- developers - lists.mariadb.org

Re: [Maria-developers] Mdev-15085 Invisible Column Non-constant Default value results...
by Sergei Golubchik 30 Jan '18

30 Jan '18

Hi, Sachin! ok to push! On Jan 30, Sachin Setiya wrote: > commit a2e6d3a2fe6842bd3c35dffb1cd6f6d8a3117107 > Author: Sachin Setiya <sachin.setiya(a)mariadb.com> > Date: Tue Jan 30 14:04:05 2018 +0530 > > Mdev-15085 Invisible Column Non-constant Default value results... > > Problem:- If we create table field with dynamic default value then that > field always gets NULL value. > > Analyze:- This is because in fill_record we simple continue at Invisible > column because we though that share->default_values(default value is > always copied into table->record[0] before insert) will have a default > value for them(which is true for constant defaults , but not for dynamic > defaults). > > Solution:- We simple set all_fields_have_value to null , and this will > make call to update_default_fields (in the case of dynamic default), And > default expr will be evaluted and value will be set in field. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] Wording suggestion?
by Sergei Golubchik 30 Jan '18

30 Jan '18

Hi, Russell! On Jan 29, Russell J.T. Dyer wrote: > Is this mode related only to UPDATE and this feature? If so, maybe it > could be something like one of these: > > update_static > update_stasis > update_isolation > > These would focus on the fact that it's related just to UPDATE and that the > values held in a state temporarily, or in isolation of each other. Sorry, I should've said earlier. It's not only for update. There is a patch to fix SELECT ... INTO, and I suppose we'll have "simultaneous assignment" behavior everywhere eventually. So, I'm leaning towards either simultaneous_assignment or simultaneous_set as mode names. The first is more exact, but it's rather long (sql_mode names range from 3 to 26 characters, and this one is 23). The second (with 16) fits right in the middle of the range, length-wise. But there's no SET keyword in SELECT ... INTO. On the other hand, there's no ASSIGNMENT keyword anywhere either :) Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 2

[Maria-developers] Wording suggestion?
by Sergei Golubchik 30 Jan '18

30 Jan '18

Hi, Ian, Russell, As you know, UPDATE in MariaDB is executed left-to-right, say, in the case of UPDATE t1 SET a=b, b=a both columns will have the same value, old value of `b`. We're now introducing a new sql_mode to have standard compatible behavior, where all assignments are executed kind of "simultaneously" (and the statement above would swap the values). What could the name for this new sql_mode be? Regards, Sergei

2 2

Re: [Maria-developers] [Commits] Mdev-15085 Rebased to latest 10.3
by Sergei Golubchik 30 Jan '18

30 Jan '18

Hi, Sachin! On Jan 29, Sachin Setiya wrote: > commit 905806689979ae91c89115f18b107ebacbc489e2 > Author: Sachin Setiya <sachin.setiya(a)maridb.com> > Date: Fri Jan 26 22:58:34 2018 +0530 > > Mdev-15085 Invisible Column Non-constant Default value results... > > Problem:- If we create table field with dynamic default value then that > field always gets NULL value. > > Analyze:- This is because in fill_record we simple continue at Invisible > column because we though that share->default_values(default value is > always copied into table->record[0] before insert) will have a default > value for them(which is true for constant defaults , but not for dynamic > defaults). > > Solution:- In case of invisible field we will assign default value to 'value' > variable, but we do this only for columns which have non dynamic default value. > We do not have to worry for Auto_increment field , storage engine will take > of it. > > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > index 45efa63..17ff7eb 100644 > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -8382,7 +8382,12 @@ fill_record(THD *thd, TABLE *table, Field **ptr, > List<Item> &values, > > if (field->invisible) > { > - continue; > + if(field->flags & AUTO_INCREMENT_FLAG || > + !field->default_value) > + continue; > + else > + value= new (thd->mem_root) Item_default_value(thd, > + &thd->lex->select_lex.context); No-no, you cannot create items in fill_record(). It would create a new item for every inserted row. I think the only thing you need to do here (and it doesn't depend on AUTO_INCREMENT_FLAG or field->default_value), you need to set all_fields_have_values to false. Like if (field->invisible) { all_fields_have_values= false; continue; } Indeed, hidden columns did not get a value, so all_fields_have_values should be false. I'd expect the rest will work automatically :) > } > else > value=v++; Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 1

Re: [Maria-developers] MDEV-14849 CREATE + ALTER with user-invisible columns produce ...
by Sergei Golubchik 29 Jan '18

29 Jan '18

Hi, Sachin! On Jan 29, Sachin Setiya wrote: > commit 5bd97f22094f3b6a658d503f98022ccd9076bb19 > Author: Sachin Setiya <sachin.setiya(a)maridb.com> > Date: Mon Jan 29 12:31:07 2018 +0530 > > MDEV-14849 CREATE + ALTER with user-invisible columns produce ... > > Problem:- > create or replace table t1 (pk int auto_increment primary key invisible, i int); > alter table t1 modify pk int invisible; > This last alter makes a invisible column which is not null and does not > have default value. > > Analysis:- > This is caused because our error check for NOT_NULL_FLAG and > NO_DEFAULT_VALUE_FLAG flag misses this sql_field , but this is not the fault > of error check :).Actually this field come via mysql_prepare_alter_table > and it does not have NO_DEFAULT_VALUE_FLAG flag turned on. (If it was create > table NO_DEFAULT_VALUE_FLAG would have turned on Column_definition::check) > and this would have generated error. > > Solution:- > I have moved the error check to kind last of mysql_prepare_create_table > because upto this point we have applied NO_DEFAULT_VALUE_FLAG to required > column. ok to push! Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] MDEV-14785 SYSTEM_INVISIBLE behaviour not consistent ---Initial Patch
by Sergei Golubchik 29 Jan '18

29 Jan '18

Hi, Sachin! On Jan 29, Sachin Setiya wrote: > commit 5e2f5aa78a5f53b000a2b9c6989c9ed7076e2527 > Author: Sachin Setiya <sachin.setiya(a)maridb.com> Note the typo in your email ^^^^^ > Date: Mon Jan 29 17:09:32 2018 +0530 > > MDEV-14785 SYSTEM_INVISIBLE behaviour not consistent > > TODO Commit message > > diff --git a/sql/item.h b/sql/item.h > index d178d9f..0072dfd 100644 > --- a/sql/item.h > +++ b/sql/item.h > @@ -2907,11 +2907,17 @@ class Item_field :public Item_ident > bool check_vcol_func_processor(void *arg) > { > context= 0; > - if (field && (field->unireg_check == Field::NEXT_NUMBER)) > + if (field) > { > - // Auto increment fields are unsupported > - return mark_unsupported_function(field_name.str, arg, VCOL_FIELD_REF | VCOL_AUTO_INC); > + if (field->unireg_check == Field::NEXT_NUMBER) > + // Auto increment fields are unsupported > + return mark_unsupported_function(field_name.str, arg, > + VCOL_FIELD_REF | VCOL_AUTO_INC); > + if (field->invisible > INVISIBLE_USER) > + return mark_unsupported_function(field_name.str, arg, > + VCOL_FIELD_REF | VCOL_SYSTEM_INVISIBLE); Why would you do that, instead of hiding system invisible fields in CREATE TABLE? From the user point of view they aren't fields, they don't exist in the table, and the error should be "no such field", not "Function or expression 'on SYSTEM_INVISIBLE column' cannot be used" > } > + > return mark_unsupported_function(field_name.str, arg, VCOL_FIELD_REF); > } > bool set_fields_as_dependent_processor(void *arg) > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > index 17ff7eb..89ba2ed 100644 > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -8128,6 +8128,11 @@ fill_record(THD *thd, TABLE *table_arg, List<Item> &fields, List<Item> &values, > value=v++; > DBUG_ASSERT(value); > Field *rfield= field->field; > + if (rfield->invisible == INVISIBLE_SYSTEM) > + { > + my_error(ER_BAD_FIELD_ERROR, MYF(0), rfield->field_name.str, thd->where); The error is correct. But do you need to do it in fill_record()? I'd thought you can do it when the statement is prepared, once, not in fill_record() which is called for every row, many times per statement. > + goto err; > + } > TABLE* table= rfield->table; > if (table->next_number_field && > rfield->field_index == table->next_number_field->field_index) > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index d2fd2ce..e4971d7 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -16639,6 +16639,7 @@ Field *create_tmp_field_from_field(THD *thd, Field *org_field, > new_field->next_equal_field= NULL; > new_field->option_list= NULL; > new_field->option_struct= NULL; > + new_field->invisible= org_field->invisible; What is it for? It'd mean that temporary tables can have invisible fields. What bugs does this change fix? > } > return new_field; > } > diff --git a/sql/sql_update.cc b/sql/sql_update.cc > index 7b4938f..7bdedc7 100644 > --- a/sql/sql_update.cc > +++ b/sql/sql_update.cc > @@ -1497,6 +1497,9 @@ int mysql_multi_update_prepare(THD *thd) > TABLE_LIST *table_list= lex->query_tables; > TABLE_LIST *tl; > List<Item> *fields= &lex->select_lex.item_list; > + List_iterator<Item> it(*fields); > + Item *item; > + Item_field *field; > table_map tables_for_update; > bool update_view= 0; > /* > @@ -1564,7 +1567,21 @@ int mysql_multi_update_prepare(THD *thd) > { > DBUG_RETURN(TRUE); > } > - > + //Check if we got any INVISIBLE_SYSTEM field > + while ((item= it++)) > + { > + if (Item::FIELD_ITEM == item->type()) > + { > + field= (Item_field *)item; > + if (field->field->invisible == INVISIBLE_SYSTEM) > + { > + my_error(ER_BAD_FIELD_ERROR,MYF(0), > + field->field->field_name.str, > + field->field->table->s->table_name.str); > + DBUG_RETURN(TRUE); > + } > + } > + } Dunno. What if I do UPDATE ... SET normal_field = row_start you'll have an invisible field, but it's not updated, so should be ok, right? May be something like if (field->field->invisible == INVISIBLE_SYSTEM && bitmap_is_set(field->table->write_set, field->field_index)) { ... my_error } And why here? I'd think this belongs into check_fields() function. > thd->table_map_for_update= tables_for_update= get_table_map(fields); Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MDEV-13790 UNHEX() of a somewhat complicated CONCAT() returns NULL
by Alexander Barkov 29 Jan '18

29 Jan '18

Hello Sergei, Please review a patch for MDEV-13790. It removes the attempt to search the best suitable buffer and just returns the result in "str" passed to val_str(). I'm quite sure this optimization gives nothing. In can be useful only under very rare conditions, but generates so many bugs. Moreover, in 10.3 (or 10.4) we'll work on reentrant items. So all Item_xxx::tmp_buffer will be gone anyway, together with this optimization. I think an optimization proposal. Why not to introduce a new method: class Item { public: .. virtual append_to_string(String *str); .. } So for example Field_varchar could write directly to "str" passed to Item_func_concat::val_str(). This will make sure that the first allocation is done on the target "str", and "str" will always be the best buffer instead of any Item_xxx::tmp_value. Thanks!

2 3

Re: [Maria-developers] 980a5c31880: MDEV-14572: Assertion `! is_set()' failed in Diagnostics_area::set_eof_status upon EXPLAIN UPDATE in PS
by Sergei Golubchik 28 Jan '18

28 Jan '18

Hi, Oleksandr! On Dec 22, Oleksandr Byelkin wrote: > revision-id: 980a5c318803d6aa7355e019869597976dcf19bf (mariadb-10.0.33-37-g980a5c31880) > parent(s): 042f763268c0f209e7c12e0a6a72bb5d204dfe29 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2017-12-22 14:00:10 +0100 > message: > > MDEV-14572: Assertion `! is_set()' failed in Diagnostics_area::set_eof_status upon EXPLAIN UPDATE in PS > > Restore EXPAIN flag in SELECT_LEX before execution multi-update by flag in LEX > (the same but in other way made before INSERT/DELETE/SELECT) Please explain what was the reason for the bug, why the status was set twice, and how your change fixes it. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] ee8755e3c51: MDEV-15012: ASAN: numerous test failures in PS
by Sergei Golubchik 26 Jan '18

26 Jan '18

Hi, Oleksandr! Nice one! Ok to push! On Jan 24, Oleksandr Byelkin wrote: > revision-id: ee8755e3c51a1da8fcf108ad0257a7e62fc94347 (mariadb-5.5.59-32-gee8755e3c51) > parent(s): 76577e1e2602f3c30859a176808c433a263e1b0a > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-01-24 14:42:52 +0100 > message: > > MDEV-15012: ASAN: numerous test failures in PS > > First roll back changes, then free Items which can lead to memory freeing. > Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] Patch for MDEV-13417 - UPDATE produces wrong values if an updated column is later used as an update source
by jerome brauge 24 Jan '18

24 Jan '18

Hello Sergei, Here is a new version of the patch following your comments. I let you decide if SET_CONSISTENCY must also be part of TRADITIONAL and ANSI. Regards, Jérôme. > -----Message d'origine----- > De : Sergei Golubchik [mailto:serg@mariadb.org] > Envoyé : jeudi 11 janvier 2018 23:17 > À : jerome brauge > Cc : bar(a)mariadb.org > Objet : Re: Patch for MDEV-13417 - UPDATE produces wrong values if an > updated column is later used as an update source > > Hi, Jérôme! > > Happy New Year! > > Sorry for the delay. > > But I've finally got around to reviewing your MDEV-13417 > > The patch was pretty good and with good tests too! > > I had only a few minor comments, see below. > If you'd like, I can apply your patch and do these changes myself. > > See the review below. > > On Dec 28, jerome brauge wrote: > > Hello Sergei, > > Do you have time to review my patch for this mdev ? > > This is very important for us because I don't see any workaround > > (without unique key and cursor). > > > diff --git a/sql/sql_class.h b/sql/sql_class.h index f30b442..a5e2104 > > 100644 > > --- a/sql/sql_class.h > > +++ b/sql/sql_class.h > > @@ -139,6 +139,7 @@ enum enum_binlog_row_image { > > #define MODE_HIGH_NOT_PRECEDENCE (1ULL << 29) > > #define MODE_NO_ENGINE_SUBSTITUTION (1ULL << 30) > > #define MODE_PAD_CHAR_TO_FULL_LENGTH (1ULL << 31) > > +#define MODE_SET_CONSISTENCY (1ULL << 32) > > I don't particularly like "SET_CONSISTENCY" name, but cannot come up with a > good alternative. It doesn't matter, renaming the mode is a trivial change > that doesn't affect anything else. > So, unless you have a strong preference for some specific mode name - just > ignore this naming issue completely. > And I'll ask our documentation guys if they could suggest something. > > > > > /* Bits for different old style modes */ > > #define OLD_MODE_NO_DUP_KEY_WARNINGS_WITH_IGNORE (1 << 0) > > diff --git a/sql/sql_base.cc b/sql/sql_base.cc index 60247c8..debb81a > > 100644 > > --- a/sql/sql_base.cc > > +++ b/sql/sql_base.cc > > @@ -7952,15 +7954,41 @@ fill_record(THD *thd, TABLE *table_arg, > List<Item> &fields, List<Item> &values, > > ER_THD(thd, > ER_WARNING_NON_DEFAULT_VALUE_FOR_VIRTUAL_COLUMN), > > rfield->field_name.str, table->s->table_name.str); > > } > > - if (rfield->stored_in_db() && > > - (value->save_in_field(rfield, 0)) < 0 && !ignore_errors) > > + if (rfield->stored_in_db()) > > { > > - my_message(ER_UNKNOWN_ERROR, ER_THD(thd, > ER_UNKNOWN_ERROR), MYF(0)); > > - goto err; > > + if (value->save_in_field(rfield, 0) < 0 && !ignore_errors) > > + { > > + my_message(ER_UNKNOWN_ERROR, ER_THD(thd, > ER_UNKNOWN_ERROR), MYF(0)); > > + goto err; > > + } > > + /* > > + ** In sql MODE_SET_CONSISTENCY, > > + ** move field pointer on value stored in record[1] > > + ** which contains row before update (see MDEV-13417) > > + */ > > + if (update && thd->variables.sql_mode & MODE_SET_CONSISTENCY) > > + rfield->move_field_offset((my_ptrdiff_t) (table->record[1] - > > + table->record[0])); > > Cool! A simple and nice solution :) > > > } > > rfield->set_explicit_default(value); > > } > > > > + if (update && thd->variables.sql_mode & MODE_SET_CONSISTENCY) { > > + // restore fields pointers on record[0] > > + f.rewind(); > > + while ((fld= f++)) > > + { > > + rfield= fld->field_for_view_update()->field; > > + if (rfield->stored_in_db()) > > + { > > + table= rfield->table; > > + rfield->move_field_offset((my_ptrdiff_t) (table->record[0] - > > + table->record[1])); > > + } > > + } > > + } > > + > > if (!update && table_arg->default_field && > > table_arg->update_default_fields(0, ignore_errors)) > > goto err; > > diff --git a/sql/sql_update.cc b/sql/sql_update.cc index > > 568dd40..5796b96 100644 > > --- a/sql/sql_update.cc > > +++ b/sql/sql_update.cc > > @@ -148,7 +148,25 @@ bool compare_record(const TABLE *table) > > we make temporary copy of Item_field, to avoid influence of changing > > result_field on Item_ref which refer on this field > > */ > > thd->change_item_tree(it.ref(), new (thd->mem_root) Item_field(thd, > field)); > > } > > + } > > + > > + if (thd->variables.sql_mode & MODE_SET_CONSISTENCY) { > > + // Make sure that a column is updated only once > > + List_iterator_fast<Item> it(items); > > + List<Item> ul; > > + ul.empty(); > > + while ((item= it++) && !ul.add_unique(item, &cmp_items)) ; > > Dunno, it's O(N²) and item->eq() is not trivial either. > perhaps it'd be faster, say, mark fields some way, like > > while ((item= it++)) > { > Field *f= item->field_for_view_update()->field; > if (f->has_explicit_value()) > { > my_error(ER_UPDATED_COLUMN_ONLY_ONCE, MYF(0), > f->table_name, f->field_name.str); > return TRUE; > } > f->set_has_explicit_value(); > } > > note that this assumes the change in the error message, %s -> %`s.%`s > > > + > > + if (item) > > + { > > + // same field appear more than once in a single update statement. > > + my_error(ER_UPDATED_COLUMN_ONLY_ONCE, MYF(0), item- > >full_name()); > > + return TRUE; > > + } > > + } > > return FALSE; > > } > > diff --git a/mysql-test/r/set_consistency.result > > b/mysql-test/r/set_consistency.result > > new file mode 100644 > > index 0000000..747b6b3 > > --- /dev/null > > +++ b/mysql-test/r/set_consistency.result > > @@ -0,0 +1,185 @@ > > +SET > > > +sql_mode='STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_A > UTO_CREA > > +TE_USER,NO_ENGINE_SUBSTITUTION,SET_CONSISTENCY'; > > shouldn't SET_CONSISTENCY be part of TRADITIONAL, ORACLE, and ANSI? > > > +# > > +# MDEV-13417 UPDATE produces wrong values if an UPDATEd column is > > +later used as an UPDATE source # MDEV-13418 Compatibility: The order > > +of evaluation of SELECT..INTO assignments # CREATE TABLE t1 (c1 > > +INTEGER, c2 INTEGER, c3 INTEGER) ENGINE=InnoDb; INSERT INTO > > +t1(c1,c2,c3) VALUES (1,1,1); CREATE TABLE t2 (c1 INTEGER, c2 > > +INTEGER, c3 INTEGER) ENGINE=InnoDb; INSERT INTO t2(c1,c2,c3) VALUES > > +(1,1,1); > > + > > +Check that a column is only updated once. > > + > > please, comment out these messages, just like the MDEV numbers above > > > +UPDATE t1 > > +SET c1 = 1, > > +c1 = 2; > > +ERROR HY000: The column test.t1.c1 cannot be changed more than once > > +in a single UPDATE statement > > Regards, > Sergei > Chief Architect MariaDB > and security(a)mariadb.org

2 1

Re: [Maria-developers] [Commits] 16298d9e82e: MDEV-7533: COLUMN_JSON() doesn't escape control characters in string values
by Sergei Golubchik 22 Jan '18

22 Jan '18

Hi, Oleksandr! On Jan 22, Oleksandr Byelkin wrote: > revision-id: 16298d9e82eb507c5e0ca6d097e6a7016582ccbc (mariadb-10.0.33-47-g16298d9e82e) > parent(s): 5e87f49a9927bf3fd0bd9af789bbdd329e4b4416 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-01-22 16:53:10 +0100 > message: > > MDEV-7533: COLUMN_JSON() doesn't escape control characters in string values > > escape all charecters less or equal 0x1F (control symbols) > (shorter sequence are not used to make code simple, long encoding is > always legal according to the rfc4627) ok to push Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MDEV-14603 signal 11 with short stacktrace
by Alexander Barkov 22 Jan '18

22 Jan '18

Hi Sergei, can you please review a patch for MDEV-14603? Thanks!

2 5

Re: [Maria-developers] [JIRA] (MDEV-14185) CREATE TEMPORARY TABLE AS SELECT causes error 1290 with read_only and InnoDB
by Sergei Golubchik 22 Jan '18

22 Jan '18

Hi, Vesa! On Jan 16, Vesa Pentti (JIRA) wrote: > > Vesa Pentti updated MDEV-14185: > ------------------------------- > Status: In Review (was: In Progress) > Assignee: Sergei Golubchik (was: Vesa Pentti) > > commit d4366c4a2c9b41901b11ca180d3ddbf79e992490 Please send commit emails next time. This time I had to go and download the patch from github and paste it in an email. > diff --git a/sql/handler.cc b/sql/handler.cc > index 174034a7eb55..85b73719f0e8 100644 > --- a/sql/handler.cc > +++ b/sql/handler.cc > @@ -4387,8 +4390,11 @@ handler::ha_create_partitioning_metadata(const char *name, const char *old_name, > partitions. > */ > DBUG_ASSERT(m_lock_type == F_UNLCK || > - (!old_name && strcmp(name, table_share->path.str))); > - mark_trx_read_write(); > + (!old_name && table_share && strcmp(name, table_share->path.str))); > + if (should_mark_rw) > + { > + mark_trx_read_write(); > + } Just remove mark_trx_read_write() from ha_create_partitioning_metadata(). It's a non-transactional operation anyway. It's always a part of DDL, so will have the same protection as DDL does. > > return create_partitioning_metadata(name, old_name, action_flag); > } Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 47bb2dcbcc8: MDEV-14786: Server crashes in Item_cond::transform on 2nd execution of SP querying from a view
by Sergei Golubchik 22 Jan '18

22 Jan '18

Hi, Oleksandr! On Jan 21, Oleksandr Byelkin wrote: > revision-id: 47bb2dcbcc8c3cdca43baf6259f2434a746c7b03 (mariadb-5.5.58-42-g47bb2dcbcc8) > parent(s): c0e964dcf1787a101f6e9c115751c0d02b0fac1b > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-01-21 23:48:32 +0100 > message: > > MDEV-14786: Server crashes in Item_cond::transform on 2nd execution of SP querying from a view > > MDEV-14957: JOIN::prepare gets unusable "conds" as argument > > Do not touch merged derived (it is irreversible) > > Fix first argument of in_optimizer for calls possible before fix_fields() > > diff --git a/sql/item.cc b/sql/item.cc > index 332e027adf1..562c0bec78d 100644 > --- a/sql/item.cc > +++ b/sql/item.cc > @@ -10010,7 +10010,7 @@ const char *dbug_print_item(Item *item) > if (!item) > return "(Item*)NULL"; > item->print(&str ,QT_ORDINARY); > - if (str.c_ptr() == buf) > + if (str.ptr() == buf) c_ptr_safe() here. It's MDEV-14981. > return buf; > else > return "Couldn't fit into buffer"; > diff --git a/sql/item.h b/sql/item.h > index 4d0860af547..add408602fc 100644 > --- a/sql/item.h > +++ b/sql/item.h > @@ -4478,4 +4478,11 @@ class Item_iterator_row: public Item_iterator > void close() {} > }; > > +#ifdef DBUG_OFF > +static inline const char *dbug_print_item(Item *item) { return NULL; } > +#else > +extern const char *dbug_print_item(Item *item); > +#endif Put it not at the end, but at the beginning - above 'class Protocol;' That's where someone had it in 10.2, it'll be easier to merge if you add it in the same place. And no 'extern' in the #else branch, just as in 10.2. > #endif /* SQL_ITEM_INCLUDED */ > diff --git a/sql/item_cmpfunc.cc b/sql/item_cmpfunc.cc > index 62e76922c0e..a40340baae9 100644 > --- a/sql/item_cmpfunc.cc > +++ b/sql/item_cmpfunc.cc > @@ -1443,6 +1443,33 @@ bool Item_in_optimizer::eval_not_null_tables(uchar *opt_arg) > } > > > +void Item_in_optimizer::print(String *str, enum_query_type query_type) > +{ > + restore_first_argumet(); > + Item_func::print(str, query_type); > +} > + > + > +/** > + "Restore" first argument before fix_fields() call (after it is harmless). > + > + @Note: Main pointer to left part of IN/ALL/ANY subselect is subselect's > + lest_expr (see Item_in_optimizer::fix_left) so changes made during > + fix_fields will be rolled back there which can make > + Item_in_optimizer::args[0] unusable on second execution before fix_left() > + call. This call fix the pointer. > +*/ > + > +void Item_in_optimizer::restore_first_argumet() > +{ > + if (args[1]->type() == Item::SUBSELECT_ITEM && > + ((Item_subselect *)args[1])->is_in_predicate()) > + { > + args[0]= ((Item_in_subselect *)args[1])->left_expr; > + } > +} Any way to trigger the bug without dbug_print_item() ? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] Mdev-14586 Assertion `0' failed in retrieve_auto_increment upon attempt to drop PK
by Sergei Golubchik 19 Jan '18

19 Jan '18

Hi, Sachin! I agree with the patch, I think it's correct. Good tests too. I wasn't able to understand the bug from the commit comment though, see below. Ok to push (preferrably with the improved commit message). On Jan 19, Sachin Setiya wrote: > Hi serg , please review the latest patch > > commit d9641478bb8f6045039a3d942f2925ada334907e > Author: Sachin Setiya <sachin.setiya(a)maridb.com> > Date: Fri Jan 19 19:33:45 2018 +0530 > > MDEV-14586 Assertion `0' failed in retrieve_auto_increment ... > > Problem:- > If we create table using myisam/aria then this crashes the server. > CREATE TABLE t1(a bit(1), b int auto_increment , index(a,b)); > insert into t1 values(1,1); > Or this query > CREATE TABLE t1 (b BIT(1), pk INTEGER AUTO_INCREMENT PRIMARY KEY); > ALTER TABLE t1 ADD INDEX(b,pk); > INSERT INTO t1 VALUES (1,b'1'); > ALTER TABLE t1 DROP PRIMARY KEY; > > Reason:- > The reason for this is > 1st- Since bit field is stored in record null bits itself , so > find_ref_key in open_binary_frm will wrongly set share->next_number_key_offset > and share->next_number_keypart (both set to 0). I would say that find_ref_key() finds what key an auto_increment field belongs to by comparing key_part->offset and field->ptr. But BIT fields might have zero length in the record, so a key might have many key parts with the same offset. That is, comparing offsets cannot uniquely identify the correct key part. > 2nd- Since next_number_key_offset is zero it myisam/aria will think that > auto_increment is in first part of key. > 3nd- myisam/aria will call retrieve_auto_key which will see first key_part > field as a bit field and call assert(0) > > Solution:- Many key parts might have the same offset, but BIT fields do not support auto_increment. So, we can skip all key parts over BIT fields, and then comparing offsets will be unambiguous. > Although the bit field is stored in record null_byte but in key file(.MYI) > this does not hold true. Bit(1-7) will take one byte of memory. So the correct > approach will be to never call retrieve_auto_increment in the case when > bit field is in the starting of index. So I have changed the find_ref_key > function to take account that if first N fields is Bit field then simply go > to next field which is not Bit field. > Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] 60d1803ec64: MDEV-6707: Wrong result (extra row) with group by, multi-part key
by Sergey Petrunia 18 Jan '18

18 Jan '18

On Thu, Jan 11, 2018 at 02:35:55AM +0530, Varun wrote: > revision-id: 60d1803ec645b6d388aac7125ac6381ecee4d548 (mariadb-5.5.56-115-g60d1803ec64) > parent(s): 9c9cf556a15af84f9133ef859b50596085f7ae8e > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-01-11 01:51:23 +0530 > message: > > MDEV-6707: Wrong result (extra row) with group by, multi-part key > > Range optimizer creates ranges according to the where conditions. So this case involves using a composite key, > few parts of which are involved in GROUP BY and few in the MIN/MAX functions in the select list. > So during the execution of such queries, we try to find a prefix using the fields involved in GROUP BY > and we then check if this newly returned prefix lies within the range we had calculated earlier. > After we find a row that satisfies the partial range(involving fields of GROUP BY), then we look at the other fields > of the where clause. We look in all the ranges if we can satisfy the above retured row. > > The issue is while calculating these prefixes we might encounter same partial ranges for multiple ranges. > An example would be > where condition is f2 ='c' AND f1 <> 9 > so the ranges would be > (NULL,c <= f1,f2 <= c,9) > (c,9 <= f1,f2 <= c,+infinity) > > In this case for calculating prefixes with the group by field we take up the > partial ranges involving field f2 those would be > c<= f2 <=c > c<= f2 <=c > So we lookup rows with the same prefix in all such ranges and then we check for the other > part(in this case f1) in all the ranges. So lets take a row of the table to be (4,'c'), that would have > a prefix 'c' and value for f1 to be 4, so it would lie in the first range mentioned above. But we would > get this same record again when we read the partial range and this would again satisfy the first range. > This issue can be fixed if we compare such partial ranges and don't lookup if we see the same prefix again. This is a huge and long comment. Please make it smaller, and move at least part of it into the code. Good places would be: - The function comment for QUICK_RANGE_SELECT::get_next_prefix - the new lines in QUICK_RANGE_SELECT::get_next_prefix which use/update the value of save_last_range > > --- > mysql-test/r/range.result | 14 ++++++++++++++ > mysql-test/r/range_mrr_icp.result | 14 ++++++++++++++ > mysql-test/t/range.test | 13 +++++++++++++ > sql/opt_range.cc | 20 +++++++++++++++++--- > sql/opt_range.h | 6 ++++-- > 5 files changed, 62 insertions(+), 5 deletions(-) > > diff --git a/mysql-test/r/range.result b/mysql-test/r/range.result > index 630a692cef6..72b31de4df4 100644 > --- a/mysql-test/r/range.result > +++ b/mysql-test/r/range.result > @@ -2144,3 +2144,17 @@ value1 1000685 12345 > value1 1003560 12345 > value1 1004807 12345 > drop table t1; > +# > +# MDEV-6707: Wrong result (extra row) with group by, multi-part key > +# > +CREATE TABLE t1 (f1 INT, f2 VARCHAR(1), KEY(f2,f1)) ENGINE=InnoDB; > +INSERT INTO t1 VALUES > +(7,'v'),(0,'s'),(9,'l'),(4,'c'); > +explain > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE t1 range f2 f2 9 NULL 2 Using where; Using index for group-by > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +MAX(f1) f2 > +4 c > +DROP TABLE t1; > diff --git a/mysql-test/r/range_mrr_icp.result b/mysql-test/r/range_mrr_icp.result > index 3f5de5b0189..1050bfcd887 100644 > --- a/mysql-test/r/range_mrr_icp.result > +++ b/mysql-test/r/range_mrr_icp.result > @@ -2146,4 +2146,18 @@ value1 1000685 12345 > value1 1003560 12345 > value1 1004807 12345 > drop table t1; > +# > +# MDEV-6707: Wrong result (extra row) with group by, multi-part key > +# > +CREATE TABLE t1 (f1 INT, f2 VARCHAR(1), KEY(f2,f1)) ENGINE=InnoDB; > +INSERT INTO t1 VALUES > +(7,'v'),(0,'s'),(9,'l'),(4,'c'); > +explain > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE t1 range f2 f2 9 NULL 2 Using where; Using index for group-by > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +MAX(f1) f2 > +4 c > +DROP TABLE t1; > set optimizer_switch=@mrr_icp_extra_tmp; > diff --git a/mysql-test/t/range.test b/mysql-test/t/range.test > index 393ca68e945..62e907b7b4a 100644 > --- a/mysql-test/t/range.test > +++ b/mysql-test/t/range.test > @@ -1718,3 +1718,16 @@ where (key1varchar='value1' AND (key2int <=1 OR key2int > 1)); > --echo # The following must show col1=12345 for all rows: > select * from t1; > drop table t1; > + > +--echo # > +--echo # MDEV-6707: Wrong result (extra row) with group by, multi-part key > +--echo # > + > +CREATE TABLE t1 (f1 INT, f2 VARCHAR(1), KEY(f2,f1)) ENGINE=InnoDB; > +INSERT INTO t1 VALUES > +(7,'v'),(0,'s'),(9,'l'),(4,'c'); > + > +explain > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +DROP TABLE t1; > diff --git a/sql/opt_range.cc b/sql/opt_range.cc > index 25a9e729a8b..e985aba1140 100644 > --- a/sql/opt_range.cc > +++ b/sql/opt_range.cc > @@ -11249,6 +11249,7 @@ int QUICK_RANGE_SELECT::get_next() > @param prefix_length length of cur_prefix > @param group_key_parts The number of key parts in the group prefix > @param cur_prefix prefix of a key to be searched for > + @save_last_range Saving the last range we encountered You've meant "@param save_last_range" ? Also please use " INOUT " to indicate this parameter is used to pass info both from the caller and to the caller. > > Each subsequent call to the method retrieves the first record that has a > prefix with length prefix_length and which is different from cur_prefix, > @@ -11271,7 +11272,8 @@ int QUICK_RANGE_SELECT::get_next() > > int QUICK_RANGE_SELECT::get_next_prefix(uint prefix_length, > uint group_key_parts, > - uchar *cur_prefix) > + uchar *cur_prefix, > + QUICK_RANGE **save_last_range) > { > DBUG_ENTER("QUICK_RANGE_SELECT::get_next_prefix"); > const key_part_map keypart_map= make_prev_keypart_map(group_key_parts); > @@ -11301,8 +11303,19 @@ int QUICK_RANGE_SELECT::get_next_prefix(uint prefix_length, > last_range= 0; > DBUG_RETURN(HA_ERR_END_OF_FILE); > } > + > last_range= *(cur_range++); > > + if (*save_last_range) > + { > + if (!key_tuple_cmp(key_part_info, (*save_last_range)->min_key, > + last_range->min_key, prefix_length)) > + { > + last_range=NULL; > + continue; > + } > + } > + *save_last_range= last_range; > key_range start_key, end_key; > last_range->make_min_endpoint(&start_key, prefix_length, keypart_map); > last_range->make_max_endpoint(&end_key, prefix_length, keypart_map); > @@ -13315,7 +13328,7 @@ QUICK_GROUP_MIN_MAX_SELECT(TABLE *table, JOIN *join_arg, bool have_min_arg, > seen_first_key(FALSE), doing_key_read(FALSE), min_max_arg_part(min_max_arg_part_arg), > key_infix(key_infix_arg), key_infix_len(key_infix_len_arg), > min_functions_it(NULL), max_functions_it(NULL), > - is_index_scan(is_index_scan_arg) > + is_index_scan(is_index_scan_arg), save_last_range(NULL) > { > head= table; > index= use_index; > @@ -13968,7 +13981,8 @@ int QUICK_GROUP_MIN_MAX_SELECT::next_prefix() > uchar *cur_prefix= seen_first_key ? group_prefix : NULL; > if ((result= quick_prefix_select->get_next_prefix(group_prefix_len, > group_key_parts, > - cur_prefix))) > + cur_prefix, > + &save_last_range))) > DBUG_RETURN(result); > seen_first_key= TRUE; > } > diff --git a/sql/opt_range.h b/sql/opt_range.h > index b8b46ae5ab1..f4fb0642590 100644 > --- a/sql/opt_range.h > +++ b/sql/opt_range.h > @@ -455,6 +455,7 @@ class QUICK_RANGE_SELECT : public QUICK_SELECT_I > QUICK_RANGE **cur_range; /* current element in ranges */ > > QUICK_RANGE *last_range; > + QUICK_RANGE *save_last_range; So, now both QUICK_RANGE_SELECT and QUICK_GROUP_MIN_MAX_SELECT have 'save_last_range'. Does QUICK_RANGE_SELECT need one? > > KEY_PART *key_parts; > KEY_PART_INFO *key_part_info; > @@ -477,7 +478,7 @@ class QUICK_RANGE_SELECT : public QUICK_SELECT_I > int get_next(); > void range_end(); > int get_next_prefix(uint prefix_length, uint group_key_parts, > - uchar *cur_prefix); > + uchar *cur_prefix, QUICK_RANGE **save_last_range); > bool reverse_sorted() { return 0; } > bool unique_key_range(); > int init_ror_merged_scan(bool reuse_handler, MEM_ROOT *alloc); > @@ -916,7 +917,8 @@ class QUICK_GROUP_MIN_MAX_SELECT : public QUICK_SELECT_I > Use index scan to get the next different key instead of jumping into it > through index read > */ > - bool is_index_scan; > + bool is_index_scan; > + QUICK_RANGE *save_last_range; > public: > /* > The following two members are public to allow easy access from BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] 6ffd535e415: MDEV-14229: Stack trace is not resolved for shared objects
by Sergei Golubchik 18 Jan '18

18 Jan '18

Hi, Vicentiu! Looks mostly ok. Does it mean, it will replace my changes in 10.0+ related to calculating the offset? struct link_map *lm = (struct link_map*) dlopen(0, RTLD_NOW); etc As far as LIBDL and dependencies are concerned... 1. Don't add LIBDL to the link list. The server has it and clients should not link with my_addr_resove (and it's good to get a linker error, if by some mistake they do) 2. Check for dladdr in configure.cmake: CHECK_FUNCTION_EXISTS (dladdr HAVE_DLADDR) 3. handle dladdr in my_global.h (search for dlopen or dlerror) But, I think, you won't need any ifdefs in my_addr_resolve.c, because my_global.h will take care of that. Ok to push after that, thanks! On Jan 16, vicentiu(a)mariadb.org wrote: > revision-id: 6ffd535e415028f414b9d1c9e9efbfb81bf25798 (mariadb-5.5.58-42-g6ffd535e415) > parent(s): 6267be460ab5147e3bc0fffd03e690b3650f2fe2 > author: Vicențiu Ciorbaru > committer: Vicențiu Ciorbaru > timestamp: 2018-01-16 22:57:52 +0200 > message: > > MDEV-14229: Stack trace is not resolved for shared objects > > Resolving a stacktrace including functions in dynamic libraries requires > us to look inside the libraries for the symbols. Addr2line needs to be > started with the correct binary for each address on the stack. To do this, > figure out which library it is using dladdr, then if the addr2line > binary was started with a different binary, fork it again with the > correct one. > > We only have one addr2line process running at any point during the > stacktrace resolving step. The maximum number of forks for addr2line should > generally be around 6. > > One for server stacktrace code, one for plugin code, one when going back > into server code, one for pthread library, one for libc, one for the > _start function in the server. More can come up if plugin calls server > function which goes back to a plugin, etc. > > --- > mysys/CMakeLists.txt | 2 +- > mysys/my_addr_resolve.c | 97 +++++++++++++++++++++++++++++++++---------------- > 2 files changed, 67 insertions(+), 32 deletions(-) > > diff --git a/mysys/CMakeLists.txt b/mysys/CMakeLists.txt > index cb86850c2de..15f3fbad9bf 100644 > --- a/mysys/CMakeLists.txt > +++ b/mysys/CMakeLists.txt > @@ -67,7 +67,7 @@ ENDIF() > > ADD_CONVENIENCE_LIBRARY(mysys ${MYSYS_SOURCES}) > TARGET_LINK_LIBRARIES(mysys dbug strings ${ZLIB_LIBRARY} > - ${LIBNSL} ${LIBM} ${LIBRT} ${LIBSOCKET} ${LIBEXECINFO}) > + ${LIBNSL} ${LIBM} ${LIBRT} ${LIBSOCKET} ${LIBEXECINFO} ${LIBDL}) > DTRACE_INSTRUMENT(mysys) > > IF(HAVE_BFD_H) > diff --git a/mysys/my_addr_resolve.c b/mysys/my_addr_resolve.c > index 90e6f43f390..f831ad5121f 100644 > --- a/mysys/my_addr_resolve.c > +++ b/mysys/my_addr_resolve.c > @@ -132,15 +132,79 @@ const char *my_addr_resolve_init() > > #include <m_string.h> > #include <ctype.h> > + > +#include <sys/wait.h> > + > static int in[2], out[2]; > -static int initialized= 0; > +static pid_t pid; > +static char addr2line_binary[1024]; > static char output[1024]; > + > +int start_addr2line_fork(const char *binary_path) > +{ > + > + if (pid > 0) > + { > + /* Don't leak FDs */ > + close(in[1]); > + close(out[0]); > + /* Don't create zombie processes. */ > + waitpid(pid, NULL, 0); > + } > + > + if (pipe(in) < 0) > + return 1; > + if (pipe(out) < 0) > + return 1; > + > + pid = fork(); > + if (pid == -1) > + return 1; > + > + if (!pid) /* child */ > + { > + dup2(in[0], 0); > + dup2(out[1], 1); > + close(in[0]); > + close(in[1]); > + close(out[0]); > + close(out[1]); > + execlp("addr2line", "addr2line", "-C", "-f", "-e", binary_path, NULL); > + exit(1); > + } > + > + close(in[0]); > + close(out[1]); > + > + return 0; > +} > + > int my_addr_resolve(void *ptr, my_addr_loc *loc) > { > char input[32], *s; > size_t len; > > - len= my_snprintf(input, sizeof(input), "%p\n", ptr); > + Dl_info info; > + void *offset; > + > + if (!dladdr(ptr, &info)) > + return 1; > + > + if (strcmp(addr2line_binary, info.dli_fname)) > + { > + /* We use dli_fname in case the path is longer than the length of our static > + string. We don't want to allocate anything dynamicaly here as we are in > + a "crashed" state. */ > + if (start_addr2line_fork(info.dli_fname)) > + { > + addr2line_binary[0] = '\0'; > + return 1; > + } > + /* Save result for future comparisons. */ > + strnmov(addr2line_binary, info.dli_fname, sizeof(addr2line_binary)); > + } > + offset = info.dli_fbase; > + len= my_snprintf(input, sizeof(input), "%p\n", ptr - offset); > if (write(in[1], input, len) <= 0) > return 1; > if (read(out[0], output, sizeof(output)) <= 0) > @@ -168,35 +232,6 @@ int my_addr_resolve(void *ptr, my_addr_loc *loc) > > const char *my_addr_resolve_init() > { > - if (!initialized) > - { > - pid_t pid; > - > - if (pipe(in) < 0) > - return "pipe(in)"; > - if (pipe(out) < 0) > - return "pipe(out)"; > - > - pid = fork(); > - if (pid == -1) > - return "fork"; > - > - if (!pid) /* child */ > - { > - dup2(in[0], 0); > - dup2(out[1], 1); > - close(in[0]); > - close(in[1]); > - close(out[0]); > - close(out[1]); > - execlp("addr2line", "addr2line", "-C", "-f", "-e", my_progname, NULL); > - exit(1); > - } > - > - close(in[0]); > - close(out[1]); > - initialized= 1; > - } > return 0; > } > #endif Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] a doubt about locks on supremum
by hongxiang jiang 18 Jan '18

18 Jan '18

2 1

Re: [Maria-developers] [Commits] 4b4267288cb: MDEV-14743: Server crashes in Item_func_match::init_search
by Sergei Golubchik 12 Jan '18

12 Jan '18

Hi, Oleksandr! Two thoughts. 1. Please use if (!ifm->fixed) then you'll only need one #ifdef. 2. How does it work with the iterator? if you remove the current element from the list and then iterator with jump to the next one, won't it cause one element to be skipped? On Jan 10, Oleksandr Byelkin wrote: > revision-id: 4b4267288cb2668717cc70537eb7e79cecaff58a (mariadb-5.5.58-24-g4b4267288cb) > parent(s): a408e881cf73d06fc92097fce6ef9584e16edf77 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-01-10 12:22:56 +0100 > message: > > MDEV-14743: Server crashes in Item_func_match::init_search > > Remove non prepared (and so belonging to removed clauses FT functions) from the list. > > in later version it will be fixed by building the list during preparation. > > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > index c06c4fcff29..26e302ded35 100644 > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -9550,7 +9550,19 @@ int init_ftfuncs(THD *thd, SELECT_LEX *select_lex, bool no_order) > DBUG_PRINT("info",("Performing FULLTEXT search")); > > while ((ifm=li++)) > - ifm->init_search(no_order); > +#if MYSQL_VERSION_ID < 100213 > + if (ifm->fixed) > +#endif > + ifm->init_search(no_order); > +#if MYSQL_VERSION_ID < 100213 > + else > + /* > + it mean that clause where was FT function was removed, so we have > + to remove the function from the list. > + */ > + li.remove(); > +#endif > + > } > return 0; > } Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 1

[Maria-developers] Eliminating or replacing the InnoDB row_prebuilt_t::fetch_cache
by Marko Mäkelä 12 Jan '18

12 Jan '18

The handler API in MySQL and MariaDB is operating on a single record at a time. There are some ‘batched’ primitives that in my understanding were developed for the NDB cluster, but generally InnoDB does not use them. The only nontrivial thing that InnoDB does during reads is the Index Condition Pushdown (ICP) and the end-of-range detection (DsMrr) that was ported from the never-released MySQL 6.0 or 5.2 to MySQL 5.6. To improve the performance of range scans, InnoDB uses a caching mechanism to prefetch results. Starting with the 4th consecutive read on the same ha_innobase handle, InnoDB would prefetch 8 records into row_prebuilt_t::fetch_cache. There are several drawbacks of this: - The parameters of this cache are hard-coded, and these caches can make wrong guesses (for SELECT * FROM t LIMIT 5, it would unnecessarily read 8 records, possibly causing unnecessary page accesses) - The caches waste memory, especially on partitioned tables, where each partition handle would have its own cache. (The main benefit of ha_innopart or InnoDB native partitioning in MySQL 5.7 is that the fetch_cache of the table is shared between partitions. It is only a benefit of saving memory, because the cache will be emptied when switching partitions.) As far as I understand, the entire purpose of the prebuilt->fetch_cache is to reduce the cost of acquiring the page latch and repositioning the cursor on each read request. If we did not call mtr_t::commit() between each request, maybe we could remove this cache altogether. When I was in the InnoDB team at MySQL, I did bring up a few times the idea of eliminating the fetch_cache, but the optimizer developers were in a separate team, and to do this, we would have needed support from both managers. I am hoping that with MariaDB this would merely be a technical challenge to first write a proof-of-concept prototype, and then to polish and test it. When I discussed this with the late Olav Sandstå some years ago, there was some thinking that we should stop hard-coding the TABLE::record[0] in various parts of the code, and the storage engine should be able to return multiple records to a larger TABLE::record[] buffer. I wonder if a simpler approach would work, and if we could do without any prefetching: 1. Remove row_sel_dequeue_cached_row_for_mysql(), prebuilt->fetch_cache, and related code. 2. Add prebuilt->mtr and let it be reused across multiple subsequent calls to row_search_mvcc(), until the query executor stops the current read batch. 3. A call to btr_pcur_store_position() and mtr_t::commit() will be necessary before any potentially long delay, such as when the result is being sent to a client connection, or when the query executor is going to switch to read records from another table. 4. The "end of read batch" could be signalled by invoking handler::extra() with a parameter. With such changes, less copying would take place, and there should be less contention on dict_index_t::lock and the upper-level buf_block_t::lock, due to fewer calls to btr_pcur_restore_position(). While that function does have a fast path ("optimistic restore"), it cannot switch to the next page without a key lookup. Range scans would be more efficient if we can always simply advance to the right leaf page and release the lock on the left page. I would love to do this in InnoDB. But I think that the effort should be lead from the optimizer side. Someone should first implement the "end of batched read" calls and some DBUG_ASSERT() that would catch any wrong-doing. That is, whenever we start to send data to a client or start to write into a temporary table, or switch to read records from another cursor, we must assert that the "end of batched read" flag is set on all cursor handles. Similarly, at the end and start of a statement, and at commit or rollback, all cursor handles must be at "end of batched read". Who would take the challenge? Best regards, Marko -- Marko Mäkelä, Lead Developer InnoDB MariaDB Corporation DON’T MISS M|18 MariaDB User Conference February 26 - 27, 2018 New York City https://m18.mariadb.com/

1 0

[Maria-developers] Check constraints
by Martijn Tonies (Upscene Productions) 10 Jan '18

10 Jan '18

Hi, Where can I find meta information for check constraints on tables? With regards, Martijn Tonies Upscene Productions http://www.upscene.com Database Workbench - developer tool for Oracle, MS SQL Server, PostgreSQL, SQL Anywhere, MySQL, InterBase, NexusDB and Firebird.

3 4

[Maria-developers] subscribe
by Martijn Tonies (Upscene Productions) 09 Jan '18

09 Jan '18

subscribe With regards, Martijn Tonies Upscene Productions http://www.upscene.com Database Workbench - developer tool for Oracle, MS SQL Server, PostgreSQL, SQL Anywhere, MySQL, InterBase, NexusDB and Firebird.

1 0

Re: [Maria-developers] [Commits] fbe0fc2: Fixed mdev-14879 Lost rows for query using recursive CTE
by Oleksandr Byelkin 09 Jan '18

09 Jan '18

Hi, Igor! Am 06.01.2018 um 18:32 schrieb IgorBabaev: > revision-id: fbe0fc2d87cb1bdb9f7040d44ad3c58a94857e20 (mariadb-10.2.12-14-gfbe0fc2) > parent(s): 15b1840f43be8c660382458826e9d5a47d148a67 > author: Igor Babaev > committer: Igor Babaev > timestamp: 2018-01-06 09:32:47 -0800 > message: > > Fixed mdev-14879 Lost rows for query using recursive CTE > with recursive reference in subquery > > If a recursive CTE uses a subquery with recursive reference then > the virtual function reset() must be called after each iteration > performed at the execution of the CTE. OK to push. [skip]

1 0

Re: [Maria-developers] [Commits] 81965f0: Fixed mdev-14852 Fails to reopen temp table within standard CTE
by Oleksandr Byelkin 05 Jan '18

05 Jan '18

Am 05.01.2018 um 08:40 schrieb IgorBabaev: > revision-id: 81965f03065b84322f2d03d784893179e8b79fd9 (mariadb-10.2.12-6-g81965f0) > parent(s): 0f253d3e644d3027face26371eb1bb59e8280728 > author: Igor Babaev > committer: Igor Babaev > timestamp: 2018-01-04 23:40:37 -0800 > message: > > Fixed mdev-14852 Fails to reopen temp table within standard CTE > > If the specification of a CTE contains a reference to a temporary table > then THD::open_temporary_table() must be called for this reference for > any occurrence of the CTE in the query. By mistake this was done only > for the first occurrences of CTEs. > The patch fixes this problem in With_element::clone_parsed_spec(). > It also moves there the call of check_dependencies_in_with_clauses() > to its proper place before the call of check_table_access(). > Additionally the patch optimizes the number of calls of the > function check_dependencies_in_with_clauses(). OK to push! > > --- > mysql-test/r/cte_nonrecursive.result | 30 ++++++++++++++++++++++++++++++ > mysql-test/t/cte_nonrecursive.test | 29 +++++++++++++++++++++++++++++ > sql/sql_cte.cc | 7 +++++++ > sql/sql_parse.cc | 22 +++++----------------- > sql/sql_prepare.cc | 11 +++-------- > 5 files changed, 74 insertions(+), 25 deletions(-) > > diff --git a/mysql-test/r/cte_nonrecursive.result b/mysql-test/r/cte_nonrecursive.result > index b1b41b1..c1f4c9f 100644 > --- a/mysql-test/r/cte_nonrecursive.result > +++ b/mysql-test/r/cte_nonrecursive.result > @@ -1265,3 +1265,33 @@ a > 4 > deallocate prepare stmt; > drop table t1; > +# > +# MDEV-14852: CTE using temporary table in query > +# with two references to the CTE > +# > +create temporary table t1 (i int); > +insert into t1 values (5),(4),(1),(2),(3); > +with > +c1 as (select i from t1), > +c2 as (select i from c1 where c1.i=2) > +select i from c1 where i > 3 union select i from c2; > +i > +5 > +4 > +2 > +drop table t1; > +create table t1 (term char(10)); > +create temporary table t2 (term char(10)); > +insert into t1 values ('TERM01'),('TERM02'),('TERM03'); > +insert into t2 values ('TERM02'),('TERM03'),('TERM04'); > +with c1 as (select * from t1), c2 as (select * from t2) > +(select * from c1 left outer join c2 on c1.term = c2.term) > +union all > +(select * from c1 right outer join c2 on c1.term = c2.term > +where c1.term is null); > +term term > +TERM02 TERM02 > +TERM03 TERM03 > +TERM01 NULL > +NULL TERM04 > +drop table t1,t2; > diff --git a/mysql-test/t/cte_nonrecursive.test b/mysql-test/t/cte_nonrecursive.test > index 1f21dbc..9436665 100644 > --- a/mysql-test/t/cte_nonrecursive.test > +++ b/mysql-test/t/cte_nonrecursive.test > @@ -853,3 +853,32 @@ execute stmt; > deallocate prepare stmt; > > drop table t1; > + > +--echo # > +--echo # MDEV-14852: CTE using temporary table in query > +--echo # with two references to the CTE > +--echo # > + > +create temporary table t1 (i int); > +insert into t1 values (5),(4),(1),(2),(3); > + > +with > +c1 as (select i from t1), > +c2 as (select i from c1 where c1.i=2) > +select i from c1 where i > 3 union select i from c2; > + > +drop table t1; > + > +create table t1 (term char(10)); > +create temporary table t2 (term char(10)); > + > +insert into t1 values ('TERM01'),('TERM02'),('TERM03'); > +insert into t2 values ('TERM02'),('TERM03'),('TERM04'); > + > +with c1 as (select * from t1), c2 as (select * from t2) > +(select * from c1 left outer join c2 on c1.term = c2.term) > +union all > +(select * from c1 right outer join c2 on c1.term = c2.term > + where c1.term is null); > + > +drop table t1,t2; > diff --git a/sql/sql_cte.cc b/sql/sql_cte.cc > index 601c192..d12948f 100644 > --- a/sql/sql_cte.cc > +++ b/sql/sql_cte.cc > @@ -817,12 +817,19 @@ st_select_lex_unit *With_element::clone_parsed_spec(THD *thd, > parse_status= parse_sql(thd, &parser_state, 0); > if (parse_status) > goto err; > + > + if (check_dependencies_in_with_clauses(lex->with_clauses_list)) > + goto err; > + > spec_tables= lex->query_tables; > spec_tables_tail= 0; > for (TABLE_LIST *tbl= spec_tables; > tbl; > tbl= tbl->next_global) > { > + if (!tbl->derived && !tbl->schema_table && > + thd->open_temporary_table(tbl)) > + goto err; > spec_tables_tail= tbl; > } > if (check_table_access(thd, SELECT_ACL, spec_tables, FALSE, UINT_MAX, FALSE)) > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index f00c74b..6d068eb 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -2990,6 +2990,9 @@ mysql_execute_command(THD *thd) > thd->get_stmt_da()->opt_clear_warning_info(thd->query_id); > } > > + if (check_dependencies_in_with_clauses(thd->lex->with_clauses_list)) > + DBUG_RETURN(1); > + > #ifdef HAVE_REPLICATION > if (unlikely(thd->slave_thread)) > { > @@ -3446,14 +3449,6 @@ mysql_execute_command(THD *thd) > ulong privileges_requested= lex->exchange ? SELECT_ACL | FILE_ACL : > SELECT_ACL; > > - /* > - The same function must be called for DML commands > - when CTEs are supported in DML statements > - */ > - res= check_dependencies_in_with_clauses(thd->lex->with_clauses_list); > - if (res) > - break; > - > if (all_tables) > res= check_table_access(thd, > privileges_requested, > @@ -3879,8 +3874,7 @@ mysql_execute_command(THD *thd) > /* Copy temporarily the statement flags to thd for lock_table_names() */ > uint save_thd_create_info_options= thd->lex->create_info.options; > thd->lex->create_info.options|= create_info.options; > - if (!(res= check_dependencies_in_with_clauses(lex->with_clauses_list))) > - res= open_and_lock_tables(thd, create_info, lex->query_tables, TRUE, 0); > + res= open_and_lock_tables(thd, create_info, lex->query_tables, TRUE, 0); > thd->lex->create_info.options= save_thd_create_info_options; > if (res) > { > @@ -4493,8 +4487,7 @@ mysql_execute_command(THD *thd) > > unit->set_limit(select_lex); > > - if (!(res= check_dependencies_in_with_clauses(lex->with_clauses_list)) && > - !(res=open_and_lock_tables(thd, all_tables, TRUE, 0))) > + if (!(res=open_and_lock_tables(thd, all_tables, TRUE, 0))) > { > MYSQL_INSERT_SELECT_START(thd->query()); > /* > @@ -4821,9 +4814,6 @@ mysql_execute_command(THD *thd) > { > List<set_var_base> *lex_var_list= &lex->var_list; > > - if (check_dependencies_in_with_clauses(thd->lex->with_clauses_list)) > - goto error; > - > if ((check_table_access(thd, SELECT_ACL, all_tables, FALSE, UINT_MAX, FALSE) > || open_and_lock_tables(thd, all_tables, TRUE, 0))) > goto error; > @@ -6376,8 +6366,6 @@ static bool execute_sqlcom_select(THD *thd, TABLE_LIST *all_tables) > new (thd->mem_root) Item_int(thd, > (ulonglong) thd->variables.select_limit); > } > - if (check_dependencies_in_with_clauses(lex->with_clauses_list)) > - return 1; > > if (!(res= open_and_lock_tables(thd, all_tables, TRUE, 0))) > { > diff --git a/sql/sql_prepare.cc b/sql/sql_prepare.cc > index be78a76..390b708 100644 > --- a/sql/sql_prepare.cc > +++ b/sql/sql_prepare.cc > @@ -1573,8 +1573,6 @@ static int mysql_test_select(Prepared_statement *stmt, > lex->select_lex.context.resolve_in_select_list= TRUE; > > ulong privilege= lex->exchange ? SELECT_ACL | FILE_ACL : SELECT_ACL; > - if (check_dependencies_in_with_clauses(lex->with_clauses_list)) > - goto error; > if (tables) > { > if (check_table_access(thd, privilege, tables, FALSE, UINT_MAX, FALSE)) > @@ -1841,9 +1839,6 @@ static bool mysql_test_create_table(Prepared_statement *stmt) > if (create_table_precheck(thd, tables, create_table)) > DBUG_RETURN(TRUE); > > - if (check_dependencies_in_with_clauses(lex->with_clauses_list)) > - DBUG_RETURN(TRUE); > - > if (select_lex->item_list.elements) > { > /* Base table and temporary table are not in the same name space. */ > @@ -2234,9 +2229,6 @@ static bool mysql_test_insert_select(Prepared_statement *stmt, > if (insert_precheck(stmt->thd, tables)) > return 1; > > - if (check_dependencies_in_with_clauses(lex->with_clauses_list)) > - return 1; > - > /* store it, because mysql_insert_select_prepare_tester change it */ > first_local_table= lex->select_lex.table_list.first; > DBUG_ASSERT(first_local_table != 0); > @@ -2339,6 +2331,9 @@ static bool check_prepared_statement(Prepared_statement *stmt) > if (tables) > thd->get_stmt_da()->opt_clear_warning_info(thd->query_id); > > + if (check_dependencies_in_with_clauses(thd->lex->with_clauses_list)) > + goto error; > + > if (sql_command_flags[sql_command] & CF_HA_CLOSE) > mysql_ha_rm_tables(thd, tables); > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

Re: [Maria-developers] [Commits] de21526: This is a full cost-based implementation of the optimization that employs
by Oleksandr Byelkin 29 Dec '17

29 Dec '17

Hi, Igor! Am 29.12.2017 um 01:47 schrieb IgorBabaev: > revision-id: de215264682a9f3291ca028fb3ae052ecb191ca3 (mariadb-10.3.3-4-gde21526) > parent(s): 7a66e0ab8f52f3bd32850463daa05f9a2401e6b1 > author: Igor Babaev > committer: Igor Babaev > timestamp: 2017-12-28 16:47:25 -0800 > message: > > This is a full cost-based implementation of the optimization that employs > splitting technique for equi-joins of materialized derived tables/views/CTEs. > (see mdev-13369 and mdev-13389). I have to make special mention of really good description. As we agreed OK to push after adding test of double execution. [skip]

1 0

Re: [Maria-developers] a01f9ff: MDEV-11071 - Assertion `thd->transaction.stmt.is_empty()' failed in
by Sergei Golubchik 27 Dec '17

27 Dec '17

Hi, Sergey! On Dec 26, Sergey Vojtovich wrote: > revision-id: a01f9ff80b892bf32dd2754da17438e8884c003a (mariadb-10.2.11-51-ga01f9ff) > parent(s): c3bd0b0301936d0dcd0918bb8c966c272610d61b > committer: Sergey Vojtovich > timestamp: 2017-12-26 14:38:45 +0400 > message: > > MDEV-11071 - Assertion `thd->transaction.stmt.is_empty()' failed in > Locked_tables_list::unlock_locked_tables > > Similarly to regular DROP TABLE, don't leave locked tables mode if CREATE OR > REPLACE dropped temporary table but failed to cerate new one. > > The problem is that there's no track of which temporary table was "locked" by > LOCK TABLES. Looks reasonable. Could you also add a test where t2 is locked and t2 is dropped? Just to show that pos_in_locked_tables works for temporary tables. Ok to push after that. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] Please document HOUR() vs EXTRACT(HOUR)
by Alexander Barkov 23 Dec '17

23 Dec '17

Hello Ian, Can you please document the difference between functions HOUR() and EXTRACT(HOUR)? Please see details here: https://jira.mariadb.org/browse/MDEV-14731 Thanks!

1 0

[Maria-developers] mysql and mariadb got different result
by 落寞回头不如华丽转身 21 Dec '17

21 Dec '17

mysql but mariadb

2 1

[Maria-developers] Fwd: Need Help
by Deepak Gupta 20 Dec '17

20 Dec '17

Hi There, I am working on a project where I have some wired requirement for my project. Issue : I am looking for the generic script that I can run with the help of Admin rights on the data base and I want the return output in such a way that it will give me the complete data base structure along with the table relation ship. For an example , there is one HR schema , with in that schema there might be 100 tables example , Employee / Salary / Address / Department etc and every table has primary and foreign key concept. So I need the script should be such a way that it will fetch all these information and as soon as I run the same script in my local system it will replicate the same data base structure for me in my local system. Kindly suggest how can I achieve this requirement ? Any help or suggestions are more than welcome. Looking foreword to here you soon. Thanks D

2 1

Re: [Maria-developers] [Commits] 203a658: MDEV-11084 Select statement with partition selection against MyISAM table opens all partitions.
by Sergei Golubchik 19 Dec '17

19 Dec '17

Hi, Alexey! On Dec 18, Alexey Botchkov wrote: > revision-id: 203a6585d3deedb36e30d5ebf388de44b6810403 (mariadb-10.1.23-51-g203a658) > parent(s): e4d10e09cf318aad237143254c45458d16009f70 > committer: Alexey Botchkov > timestamp: 2017-12-18 19:31:26 +0400 > message: > > MDEV-11084 Select statement with partition selection against MyISAM table opens all partitions. > > Now SELECT FROM t PARTITION(x) only opens the 'x' file. > The table->partition_names parameter is sent to ha_open > so it only opens the required partitions. > If the table is reopened, the > change_partition_to_open(partition_names) is called that > closes and opens partitions specified for the query. Looks ok to me, thanks! Please add some tests, can status variables show the difference in the number of opened partitions? And make sure that the whole tree compiles without partitioning. Then ok to push into 10.3 Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] 7a7ad82: MDEV-13478 Full SST sync fails because of the error in the cleaning part
by Sergei Golubchik 19 Dec '17

19 Dec '17

Hi, Sachin! Unfortunately, your comment is rather difficult to understand. What about this one: Problem: The command was: find $paths -mindepth 1 -regex $cpat -prune -o -exec rm -rf {} \+ Which was supposed to work as * skipping $paths directories themselves (-mindepth 1) * see if the dir/file name matches $cpat (-regex) * if yes - don't dive into the directory, skip it (-prune) * otherwise (-o) * remove it and everything inside (-exec) Now -exec ... \+ works like this: every new found path is appended to the end of the command line. when accumulated command line length reaches `getconf ARG_MAX` (~2Gb) it's executed, and find continues, appending to a new command line. What happens here, find appends some directory to the command line, then dives into it, and starts appending files from that directory. At some point command line overflows, rm -rf gets executed and removes the whole directory. Now find tries to continue scanning the directory that was already removed. Fix: don't dive into directories that will be recursively removed anyway, use -prune for them. Basically, we should be pruning both paths that have matched $cpat and paths that have not matched it. This is achived by pruning unconditionally, before the regex is tested: find $paths -mindepth 1 -prune -regex $cpat -o -exec rm -rf {} \+ On Dec 19, sachin wrote: > revision-id: 7a7ad82029a6c78d31d6736a562c12d02c4d968c (mariadb-galera-5.5.58-3-g7a7ad82) > parent(s): e6e026ae51a77969749de201d491a176483bbc69 > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2017-12-19 22:30:43 +0530 > message: > > MDEV-13478 Full SST sync fails because of the error in the cleaning part > > Problem:- The problem is in wsrep_sst_xtrabackup-v2.sh we use > find $ib_home_dir $ib_log_dir $ib_undo_dir $DATA -mindepth 1 -regex $cpat -prune -o -exec rm -rfv {} 1>&2 \+ > the problem is that since we have '\+' in end that means all output > will be expanded after rm -rfv . If we have really large database( > quite a no of tables with big names) then this create a problem. > This will result in calling 'rm -rvf' two time(or may be more). So non > deterministicly this might that upto directory name went to rm and > remaining was truncated. For example consider a folder xyz with lots > of files. We executed > find xyz -exec rm -rfv {} \+ > Since we have like millions of file in rm, So it will be greater then > ARG_MAX , so there will be multiple rm invocation. Say in nth invocation > this might happen rm ....... xyz/ {So it get truncated at xyz/} > Above will remove the whole xyz directory and remaining invocation > of rm will return error. How ever this type of error is non deterministic > so is bug. > Solution:- > In above example if instead of removing each file in xyz if we remove > xyz then we have our solution :). > Actually if we shift the -prune term in find we can get to solution. > Why currently find is working like this > find ( -regex && -prune) || -exec > that means if -regex is true (-prune will always return true) then exec > (and hence rm )wont work. But if regex fails then exec is applied with > out -prune which makes rm delete each single file instead of folder. > So we change the position of regex and prune then prune will always be > applied whether regex is true or not. > find (-prune && -regex) || -exec > > Patch Credit:- Serg > > --- > scripts/wsrep_sst_xtrabackup-v2.sh | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/scripts/wsrep_sst_xtrabackup-v2.sh b/scripts/wsrep_sst_xtrabackup-v2.sh > index 327c92f..00d8fe2 100644 > --- a/scripts/wsrep_sst_xtrabackup-v2.sh > +++ b/scripts/wsrep_sst_xtrabackup-v2.sh > @@ -863,7 +863,7 @@ then > > > wsrep_log_info "Cleaning the existing datadir and innodb-data/log directories" > - find $ib_home_dir $ib_log_dir $ib_undo_dir $DATA -mindepth 1 -regex $cpat -prune -o -exec rm -rfv {} 1>&2 \+ > + find $ib_home_dir $ib_log_dir $ib_undo_dir $DATA -mindepth 1 -prune -regex $cpat -o -exec rm -rfv {} 1>&2 \+ > > tempdir=$(parse_cnf mysqld log-bin "") > if [[ -n ${tempdir:-} ]];then Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 1

Re: [Maria-developers] 665402a: MDEV-14265 - RPMLint warning: shared-lib-calls-exit
by Sergei Golubchik 18 Dec '17

18 Dec '17

Hi, Sergey! On Dec 18, Sergey Vojtovich wrote: > revision-id: 665402aa9b83565c1cef3caa06cd998d3808f0d7 (mariadb-10.1.29-31-g665402a) > parent(s): ef9e78c9d41a5ac644f08068e3dabad948b0e30a > committer: Sergey Vojtovich > timestamp: 2017-12-18 20:23:36 +0400 > message: > > MDEV-14265 - RPMLint warning: shared-lib-calls-exit > > find_type_with_warning() client helper did exit(1) on error, exit(1) moved to > clients. It was find_type_or_exit(), and you could've had less changes if you'd replaced it with an inline function. Basically, copied the function into the header and made it static inline. > mysql_read_default_options() did exit(1) on error, error is passed through and > handled now. ok > my_str_malloc_default() did exit(1) on error, replaced my_str_ allocator > functions with normal my_malloc()/my_realloc()/my_free(). I think you need to replace my_str_malloc_default() with my_malloc(..., MYF(MY_FAE)), not my_malloc(..., MYF(0)) > sql_connect.cc did many exit(1) on hash initialisation failure. Removed error > check since my_hash_init() never fails. > > my_malloc() did exit(1) on error. Replaced with abort(). > > my_load_defaults() did exit(1) on error, replaced with abort(). I'd say, return 2 instead of exit/abort. my_load_defaults already does it, because my_search_option_files returns 2 for OOM. > my_load_defaults() still does exit(0) when invoked with --print-defaults. dunno, return -1, for example, and exit in the caller? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 1

[Maria-developers] MDEV-14031 Password policy causes replication failure
by Alexander Barkov 18 Dec '17

18 Dec '17

Hi Sergei, Please review a patch for MDEV-14031. Thanks!

1 0

Re: [Maria-developers] [Commits] 2603dc0: MDEV-11084 Select statement with partition selection against MyISAM table opens all partitions.
by Sergei Golubchik 15 Dec '17

15 Dec '17

Hi, Alexey! That looked reasonable enough. I didn't find anything big. Small comments - see below. On Jun 01, Alexey Botchkov wrote: > revision-id: 2603dc0d56ec90f308678a3c002f1b75af69f554 (mariadb-10.1.23-51-g2603dc0) > parent(s): e4d10e09cf318aad237143254c45458d16009f70 > committer: Alexey Botchkov > timestamp: 2017-06-01 17:33:34 +0400 > message: > > MDEV-11084 Select statement with partition selection against MyISAM table opens all partitions. > > Now SELECT FROM t PARTITION(x) only opens the 'x' file. > The table->partition_names parameter is sent to ha_open > so it only opens the required partitions. > If the table is reopened, the > change_partition_to_open(partition_names) is called that > closes and opens partitions specified for the query. > > diff --git a/sql/ha_partition.cc b/sql/ha_partition.cc > index 29054c7..53a128c 100644 > --- a/sql/ha_partition.cc > +++ b/sql/ha_partition.cc > @@ -3481,12 +3498,16 @@ int ha_partition::open(const char *name, int mode, uint test_if_locked) > } > name_buffer_ptr+= strlen(name_buffer_ptr) + 1; > } > + file_sample= m_file[0]; are you sure that m_file[0] is opened? > } > else > { > file= m_file; > do > { > + if (!bitmap_is_set(&m_opened_partitions, file - m_file)) > + continue; > + > create_partition_name(name_buff, name, name_buffer_ptr, NORMAL_PART_NAME, > FALSE); > table->s->connect_string = m_connect_string[(uint)(file-m_file)]; > @@ -3494,20 +3515,21 @@ int ha_partition::open(const char *name, int mode, uint test_if_locked) > test_if_locked | HA_OPEN_NO_PSI_CALL))) > goto err_handler; > bzero(&table->s->connect_string, sizeof(LEX_STRING)); > - if (m_file == file) > - m_num_locks= (*file)->lock_count(); > + if (!file_sample) > + m_num_locks= (file_sample= (*file))->lock_count(); > DBUG_ASSERT(m_num_locks == (*file)->lock_count()); > - name_buffer_ptr+= strlen(name_buffer_ptr) + 1; > - } while (*(++file)); > + } while (name_buffer_ptr+= strlen(name_buffer_ptr) + 1, *(++file)); why? > } > > file= m_file; > - ref_length= (*file)->ref_length; > - check_table_flags= (((*file)->ha_table_flags() & > + ref_length= file_sample->ref_length; > + check_table_flags= ((file_sample->ha_table_flags() & > ~(PARTITION_DISABLED_TABLE_FLAGS)) | > (PARTITION_ENABLED_TABLE_FLAGS)); > while (*(++file)) > { > + if (!bitmap_is_set(&m_opened_partitions, file - m_file)) > + continue; > /* MyISAM can have smaller ref_length for partitions with MAX_ROWS set */ > set_if_bigger(ref_length, ((*file)->ref_length)); > /* > @@ -6782,6 +6809,63 @@ void ha_partition::get_dynamic_partition_info(PARTITION_STATS *stat_info, > } > > > +void ha_partition::set_partitions_to_open(List<String> *partition_names) > +{ > + m_partitions_to_open= partition_names; > +} > + > + > +int ha_partition::change_partitions_to_open(List<String> *partition_names) > +{ > + char name_buff[FN_REFLEN]; > + handler **file; > + char *name_buffer_ptr; > + int error= 0; > + > + if (m_is_clone_of) > + return 0; > + > + m_partitions_to_open= partition_names; > + if ((error= m_part_info->set_partition_bitmaps(partition_names))) > + goto err_handler; > + > + if (bitmap_cmp(&m_opened_partitions, &m_part_info->read_partitions) != 0) > + return 0; > + > + if ((error= read_par_file(table->s->normalized_path.str))) > + goto err_handler; > + > + name_buffer_ptr= m_name_buffer_ptr; > + file= m_file; > + do > + { > + int n_file= file-m_file; > + int is_open= bitmap_is_set(&m_opened_partitions, n_file); > + int should_be_open= bitmap_is_set(&m_part_info->read_partitions, n_file); > + > + if (is_open && !should_be_open) > + (*file)->ha_close(); > + else if (!is_open && should_be_open) > + { > + create_partition_name(name_buff, table->s->normalized_path.str, > + name_buffer_ptr, NORMAL_PART_NAME, FALSE); > + table->s->connect_string = m_connect_string[(uint)(file-m_file)]; > + if ((error= (*file)->ha_open(table, name_buff, m_mode, > + m_open_test_lock | HA_OPEN_NO_PSI_CALL))) > + goto err_handler; > + bzero(&table->s->connect_string, sizeof(LEX_STRING)); > + } > + } while (name_buffer_ptr+= strlen(name_buffer_ptr) + 1, *(++file)); Could you reuse that, instead of copying from ::open? Perhaps ::open() can reset the bitmap to be empty and then call ::change_partitions_to_open() ? Or the common code could be moved into a third function that these both will use... > + > + bitmap_copy(&m_opened_partitions, &m_part_info->read_partitions); > + > + clear_handler_file(); > + > +err_handler: > + return error; > +} > + > + > /** > General function to prepare handler for certain behavior. > > diff --git a/sql/partition_info.cc b/sql/partition_info.cc > index bc0db9e..7fb8f70 100644 > --- a/sql/partition_info.cc > +++ b/sql/partition_info.cc > @@ -283,6 +281,27 @@ bool partition_info::set_partition_bitmaps(TABLE_LIST *table_list) > > > /** > + Set read/lock_partitions bitmap over non pruned partitions > + > + @param table_list Possible TABLE_LIST which can contain > + list of partition names to query > + > + @return Operation status > + @retval FALSE OK > + @retval TRUE Failed to allocate memory for bitmap or list of partitions > + did not match > + > + @note OK to call multiple times without the need for free_bitmaps. > +*/ > +bool partition_info::set_partition_bitmaps_from_table(TABLE_LIST *table_list) > +{ > + List<String> *partition_names= table_list ? > + NULL : table_list->partition_names; > + return set_partition_bitmaps(partition_names); > +} This looks unused. > + > + > +/** > Checks if possible to do prune partitions on insert. > > @param thd Thread context > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > index 9c56b7d..2f9ec3e 100644 > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -2533,6 +2533,12 @@ bool open_table(THD *thd, TABLE_LIST *table_list, Open_table_context *ot_ctx) > { > DBUG_ASSERT(table->file != NULL); > MYSQL_REBIND_TABLE(table->file); > + if (table->part_info) > + { > + /* Set all [named] partitions as used. */ > + if (table->file->change_partitions_to_open(table_list->partition_names)) > + DBUG_RETURN(true); > + } > } > else > { > @@ -2549,7 +2555,8 @@ bool open_table(THD *thd, TABLE_LIST *table_list, Open_table_context *ot_ctx) > HA_TRY_READ_ONLY), > (READ_KEYINFO | COMPUTE_TYPES | > EXTRA_RECORD), > - thd->open_options, table, FALSE); > + thd->open_options, table, FALSE, > + table_list->partition_names); Please, make sure you code compiles without partitioning too. (in this case I wouldn't make the last argument of open_table_from_share conditional, but I'd rather make table_list->partition_names unconditional). > > if (error) > { > @@ -2598,9 +2605,11 @@ bool open_table(THD *thd, TABLE_LIST *table_list, Open_table_context *ot_ctx) > #ifdef WITH_PARTITION_STORAGE_ENGINE > if (table->part_info) > { > +#ifdef CODE_TO_REMOVE > /* Set all [named] partitions as used. */ > if (table->part_info->set_partition_bitmaps(table_list)) > DBUG_RETURN(true); > +#endif /*CODE_TO_REMOVE*/ why not to remove at once? > } > else if (table_list->partition_names) > { Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 14acb68: MDEV-14593 human-readable XA RECOVER.
by Sergei Golubchik 14 Dec '17

14 Dec '17

Hi, Alexey! On Dec 12, Alexey Botchkov wrote: > revision-id: 14acb687199f0d30444f8c0849c0d1c1284f3e2b (mariadb-10.2.11-26-g14acb68) > parent(s): 4d016e6ed2d32298977a66e125cbfcae39e23847 > committer: Alexey Botchkov > timestamp: 2017-12-12 14:31:41 +0400 > message: > > MDEV-14593 human-readable XA RECOVER. > > The 'data' field in the XA RECOVER resultset changed > to be charset_bin. It seems to me right and also > --binary-as-hex starts working. > The XA RECOVER FORMAT='SQL' option implemented. > It returns the XID string that fits to be an argument for the > XA ... statements. Looks ok, just a couple of comments: > +xa recover format='SQL'; Please add a test for format=SQL (without quotes). We normally document EXPLAIN FORMAT=JSON without quotes (although it works with quotes too), so XA RECOVER should work the same way. > +ER_UNKNOWN_XID_FORMAT > + eng "Unknown XID format name: '%s'" Don't add a new error message, reuse ER_UNKNOWN_EXPLAIN_FORMAT: - eng "Unknown EXPLAIN format name: '%s'" - rus "Неизвестное имя формата команды EXPLAIN: '%s'" + eng "Unknown %s format name: '%s'" + rus "Неизвестное имя формата команды %s: '%s'" note, you'll need either to remove/rephrase the russian error message, or use "XA RECOVER" as the first argument, not "XID". Ok to push (into bb-10.2-ext) with these changes. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] 8048670: MDEV-10715 Galera: Replicate MariaDB GTID to other nodes in the cluster
by Jan Lindström 12 Dec '17

12 Dec '17

On Mon, Dec 11, 2017 at 9:40 AM, sachin <sachin.setiya(a)mariadb.com> wrote: > revision-id: 8048670e25e5cda915ad7ce16dbd66afa6227aa5 > (mariadb-10.1.29-24-g8048670) > parent(s): de76cbdcb0728b80c0f7b453b24b0b81f653e450 > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2017-12-11 13:05:08 +0530 > message: > > MDEV-10715 Galera: Replicate MariaDB GTID to other nodes in the cluster > > Problem:- Gtid are not transferred in Galera Cluster. > > Solution:- We need to transfer gtid in the case on either when cluster is > slave/master in async replication. In normal Gtid replication gtid are > generated on > recieving node itself and it is always on sync with other nodes. Because > galera keeps > node in sync , So all nodes get same no of event groups. So the issue > arises when > say galera is slave in async replication. > A > | (Async replication) > D <-> E <-> F {Galera replication} > So what should happen is that all node should apply the master gtid but > this does > node happen, becuase node E, F does not recieve gtid from D in write set , > So what E(or F) > does is that it applies wsrep_gtid_domain_id, D server-id , E gtid next > seq no. This > generated gtid does not always work when say A has different domain id. > > So In this commit, on galera node when we see that this event is recieved > from master > we simply write Gtid_Log_Event in write_set and send it to other nodes. > > --- > mysql-test/suite/galera/r/galera_gtid_slave.result | 26 +++ > .../r/galera_gtid_slave_sst_mysqldump.result | 133 +++++++++++++ > .../galera/r/galera_gtid_slave_sst_rsync.result | 130 +++++++++++++ > mysql-test/suite/galera/t/galera_gtid_slave.cnf | 18 ++ > mysql-test/suite/galera/t/galera_gtid_slave.test | 78 ++++++++ > .../galera/t/galera_gtid_slave_sst_mysqldump.cnf | 19 ++ > .../galera/t/galera_gtid_slave_sst_mysqldump.test | 208 > +++++++++++++++++++++ > .../suite/galera/t/galera_gtid_slave_sst_rsync.cnf | 18 ++ > .../galera/t/galera_gtid_slave_sst_rsync.test | 207 > ++++++++++++++++++++ > mysql-test/suite/galera_3nodes/galera_2x3nodes.cnf | 122 ++++++++++++ > .../galera_3nodes/r/galera_gtid_2_cluster.result | 117 ++++++++++++ > .../galera_3nodes/t/galera_gtid_2_cluster.cnf | 28 +++ > .../galera_3nodes/t/galera_gtid_2_cluster.test | 143 ++++++++++++++ > sql/log.cc | 51 ++++- > sql/wsrep_mysqld.cc | 10 + > 15 files changed, 1306 insertions(+), 2 deletions(-) > > diff --git a/mysql-test/suite/galera/r/galera_gtid_slave.result > b/mysql-test/suite/galera/r/galera_gtid_slave.result > new file mode 100644 > index 0000000..40f3f1c > --- /dev/null > +++ b/mysql-test/suite/galera/r/galera_gtid_slave.result > @@ -0,0 +1,26 @@ > +START SLAVE; > +CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB; > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB; > +INSERT INTO t1 VALUES(1); > +begin; > +insert into t2 values(21); > +insert into t2 values(22); > +commit; > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4 > +INSERT INTO t1 VALUES(2); > +INSERT INTO t1 VALUES(3); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-2 > +INSERT INTO t1 VALUES(4); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-2,2-3-3 > +DROP TABLE t1,t2; > +reset master; > +STOP SLAVE; > +RESET SLAVE ALL; > +reset master; > +reset master; > diff --git a/mysql-test/suite/galera/r/galera_gtid_slave_sst_mysqldump.result > b/mysql-test/suite/galera/r/galera_gtid_slave_sst_mysqldump.result > new file mode 100644 > index 0000000..78a0a17 > --- /dev/null > +++ b/mysql-test/suite/galera/r/galera_gtid_slave_sst_mysqldump.result > @@ -0,0 +1,133 @@ > +#Connection 2 > +START SLAVE; > +include/wait_for_slave_to_start.inc > +#Connection 1 > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY, f2 int unique) ENGINE=InnoDB; > +INSERT INTO t2 VALUES(1,11); > +INSERT INTO t2 VALUES(2,22); > +INSERT INTO t2 VALUES(3,33); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4 > +include/save_master_gtid.inc > +#Connection 2 > +include/sync_with_master_gtid.inc > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4 > +INSERT INTO t2 VALUES(4,44); > +INSERT INTO t2 VALUES(5,55); > +INSERT INTO t2 VALUES(6,66); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-3 > +#Connection 3 > +INSERT INTO t2 VALUES(7,77); > +INSERT INTO t2 VALUES(8,88); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-3,2-3-5 > +#Connection 1 > +CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_before'); > +INSERT INTO t1 VALUES ('node1_committed_before'); > +COMMIT; > +include/save_master_gtid.inc > +#Connection 2 > +include/sync_with_master_gtid.inc > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node2_committed_before'); > +INSERT INTO t1 VALUES ('node2_committed_before'); > +COMMIT; > +#Connection 3 > +Shutting down server ... > +#Connection 2 > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_during'); > +INSERT INTO t1 VALUES ('node1_committed_during'); > +COMMIT; > +#Connection 3 > +Starting server ... > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node3_committed_after'); > +INSERT INTO t1 VALUES ('node3_committed_after'); > +COMMIT; > +#Connection 2 > +Select * from t1 order by f1; > +f1 > +node1_committed_before > +node1_committed_before > +node1_committed_during > +node1_committed_during > +node2_committed_before > +node2_committed_before > +node3_committed_after > +node3_committed_after > +#Connection 3 > +Select * from t1 order by f1; > +f1 > +node1_committed_before > +node1_committed_before > +node1_committed_during > +node1_committed_during > +node2_committed_before > +node2_committed_before > +node3_committed_after > +node3_committed_after > +#Connection 2 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-2-7,2-3-8 > +#Connection 3 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-2-7,2-3-8 > +#Connection 1 > +SET AUTOCOMMIT=ON; > +#Connection 2 > +SET AUTOCOMMIT=ON; > +#Connection 3 > +SET AUTOCOMMIT=ON; > +#Connection 2 > +STOP slave; > +include/wait_for_slave_to_stop.inc > +INSERT INTO t1 VALUES ('node2_slave_stoped'); > +#Connection 1 > +INSERT INTO t1 VALUES ('node1_normal_entry'); > +include/save_master_gtid.inc > +#Connection 2 > +INSERT INTO t1 VALUES ('node2_slave_stoped_inserted'); > +start slave; > +include/wait_for_slave_to_start.inc > +include/sync_with_master_gtid.inc > +INSERT INTO t1 VALUES ('node2_slave_started'); > +SELECT count(*) from t1; > +count(*) > +12 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-7,2-3-8,2-2-11 > +#Connection 3 > +SELECT count(*) from t1; > +count(*) > +12 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-7,2-3-8,2-2-11 > +#Connection 1 > +DROP TABLE t2,t1; > +#Connection 2 > +#Connection 3 > +#Connection 2 > +STOP SLAVE; > +RESET SLAVE; > +reset master; > +#Connection 3 > +reset master; > +#Connection 1 > +reset master; > diff --git a/mysql-test/suite/galera/r/galera_gtid_slave_sst_rsync.result > b/mysql-test/suite/galera/r/galera_gtid_slave_sst_rsync.result > new file mode 100644 > index 0000000..81fae57 > --- /dev/null > +++ b/mysql-test/suite/galera/r/galera_gtid_slave_sst_rsync.result > @@ -0,0 +1,130 @@ > +#Connection 2 > +START SLAVE; > +#Connection 1 > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY, f2 int unique) ENGINE=InnoDB; > +INSERT INTO t2 VALUES(1,11); > +INSERT INTO t2 VALUES(2,22); > +INSERT INTO t2 VALUES(3,33); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4 > +include/save_master_gtid.inc > +#Connection 2 > +include/sync_with_master_gtid.inc > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4 > +INSERT INTO t2 VALUES(4,44); > +INSERT INTO t2 VALUES(5,55); > +INSERT INTO t2 VALUES(6,66); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-3 > +#Connection 3 > +INSERT INTO t2 VALUES(7,77); > +INSERT INTO t2 VALUES(8,88); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-3,2-3-5 > +#Connection 1 > +CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_before'); > +INSERT INTO t1 VALUES ('node1_committed_before'); > +COMMIT; > +include/save_master_gtid.inc > +#Connection 2 > +include/sync_with_master_gtid.inc > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node2_committed_before'); > +INSERT INTO t1 VALUES ('node2_committed_before'); > +COMMIT; > +#Connection 3 > +Shutting down server ... > +#Connection 2 > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_during'); > +INSERT INTO t1 VALUES ('node1_committed_during'); > +COMMIT; > +#Connection 3 > +Starting server ... > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node3_committed_after'); > +INSERT INTO t1 VALUES ('node3_committed_after'); > +COMMIT; > +#Connection 2 > +Select * from t1 order by f1; > +f1 > +node1_committed_before > +node1_committed_before > +node1_committed_during > +node1_committed_during > +node2_committed_before > +node2_committed_before > +node3_committed_after > +node3_committed_after > +#Connection 3 > +Select * from t1 order by f1; > +f1 > +node1_committed_before > +node1_committed_before > +node1_committed_during > +node1_committed_during > +node2_committed_before > +node2_committed_before > +node3_committed_after > +node3_committed_after > +#Connection 2 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-2-7,2-3-8 > +#Connection 3 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-2-7,2-3-8 > +#Connection 1 > +SET AUTOCOMMIT=ON; > +#Connection 2 > +SET AUTOCOMMIT=ON; > +#Connection 3 > +SET AUTOCOMMIT=ON; > +#Connection 2 > +STOP slave; > +INSERT INTO t1 VALUES ('node2_slave_stoped'); > +#Connection 1 > +INSERT INTO t1 VALUES ('node1_normal_entry'); > +include/save_master_gtid.inc > +#Connection 2 > +INSERT INTO t1 VALUES ('node2_slave_stoped_inserted'); > +start slave; > +include/sync_with_master_gtid.inc > +INSERT INTO t1 VALUES ('node2_slave_started'); > +SELECT count(*) from t1; > +count(*) > +12 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-7,2-3-8,2-2-11 > +#Connection 3 > +SELECT count(*) from t1; > +count(*) > +12 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-7,2-3-8,2-2-11 > +#Connection 1 > +DROP TABLE t2,t1; > +#Connection 2 > +#Connection 3 > +#Connection 2 > +STOP SLAVE; > +RESET SLAVE ALL; > +reset master; > +#Connection 3 > +reset master; > +#Connection 1 > +reset master; > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave.cnf > b/mysql-test/suite/galera/t/galera_gtid_slave.cnf > new file mode 100644 > index 0000000..409d0d1 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave.cnf > @@ -0,0 +1,18 @@ > +!include ../galera_2nodes_as_slave.cnf > + > +[mysqld] > +log-bin=mysqld-bin > +log-slave-updates > +binlog-format=ROW > + > +[mysqld.1] > +gtid-domain-id=1 > +[mysqld.2] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > +[mysqld.3] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > + > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave.test > b/mysql-test/suite/galera/t/galera_gtid_slave.test > new file mode 100644 > index 0000000..8ef8745 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave.test > @@ -0,0 +1,78 @@ > +# > +# Test Galera as a slave to a MariaDB master using GTIDs > +# > +# suite/galera/galera_2nodes_as_slave.cnf describes the setup of the > nodes > +# suite/galera/t/galera_as_slave_gtid.cnf has the GTID options > +# > +# In addition to performing DDL and DML, we check that the gtid of the > master is preserved inside the cluster > +# > + > +--source include/have_innodb.inc > + > +# As node #1 is not a Galera node, we connect to node #2 in order to run > include/galera_cluster.inc > +--connect node_2a, 127.0.0.1, root, , test, $NODE_MYPORT_2 > +--source include/galera_cluster.inc > + > +--connection node_2 > +--disable_query_log > +--eval CHANGE MASTER TO MASTER_HOST='127.0.0.1', MASTER_USER='root', > MASTER_PORT=$NODE_MYPORT_1; > +--enable_query_log > +START SLAVE; > + > +--connection node_1 > +CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB; > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB; > +INSERT INTO t1 VALUES(1); > + > +#multi stmt trans > +begin; > +insert into t2 values(21); > +insert into t2 values(22); > +commit; > + > +SELECT @@global.gtid_binlog_state; > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 1 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--let $wait_condition = SELECT COUNT(*) = 1 FROM t1; > +--source include/wait_condition.inc > +--sleep 1 > +INSERT INTO t1 VALUES(2); > +INSERT INTO t1 VALUES(3); > +SELECT @@global.gtid_binlog_state; > + > +--connect node_3, 127.0.0.1, root, , test, $NODE_MYPORT_3 > +--let $wait_condition = SELECT COUNT(*) = 3 FROM t1; > +--source include/wait_condition.inc > + > +INSERT INTO t1 VALUES(4); > +SELECT @@global.gtid_binlog_state; > + > +--connection node_1 > +DROP TABLE t1,t2; > +reset master; > +# > +# Unfortunately without the sleep below the following statement fails > with "query returned no rows", which > +# is difficult to understand given that it is an aggregate query. A > "query execution was interrupted" > +# warning is also reported by MTR, which is also weird. > +# > + > +--sleep 1 > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--connection node_2 > +STOP SLAVE; > +RESET SLAVE ALL; > +reset master; > + > +--connection node_3 > +reset master; > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.cnf > b/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.cnf > new file mode 100644 > index 0000000..0b4a776 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.cnf > @@ -0,0 +1,19 @@ > +!include ../galera_2nodes_as_slave.cnf > + > +[mysqld] > +log-bin=mysqld-bin > +log-slave-updates > +binlog-format=ROW > +wsrep_sst_method=xtrabackup > + > +[mysqld.1] > +gtid-domain-id=1 > +[mysqld.2] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > +[mysqld.3] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > + > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.test > b/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.test > new file mode 100644 > index 0000000..b507698 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.test > @@ -0,0 +1,208 @@ > +# > +# Test Galera as a slave to a MariaDB master using GTIDs > +# > +# suite/galera/galera_2nodes_as_slave.cnf describes the setup of the > nodes > +# > +# In addition to performing DDL and DML, we check that the gtid of the > master is preserved inside the cluster > +# > + > +--source include/big_test.inc > +--source include/have_innodb.inc > +# As node #1 is not a Galera node, we connect to node #2 in order to run > include/galera_cluster.inc > +--connect node_2a, 127.0.0.1, root, , test, $NODE_MYPORT_2 > +--source include/galera_cluster.inc > + > +--echo #Connection 2 > +--connection node_2 > +--disable_query_log > +--eval CHANGE MASTER TO MASTER_HOST='127.0.0.1', MASTER_USER='root', > MASTER_PORT=$NODE_MYPORT_1,master_use_gtid=slave_pos; > +--enable_query_log > +START SLAVE; > +--source include/wait_for_slave_to_start.inc > + > +--echo #Connection 1 > +--connection node_1 > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY, f2 int unique) ENGINE=InnoDB; > +INSERT INTO t2 VALUES(1,11); > +INSERT INTO t2 VALUES(2,22); > +INSERT INTO t2 VALUES(3,33); > + > +SELECT @@global.gtid_binlog_state; > +--source include/save_master_gtid.inc > + > +--echo #Connection 2 > +--connection node_2 > +--source include/sync_with_master_gtid.inc > +SELECT @@global.gtid_binlog_state; > +INSERT INTO t2 VALUES(4,44); > +INSERT INTO t2 VALUES(5,55); > +INSERT INTO t2 VALUES(6,66); > +SELECT @@global.gtid_binlog_state; > + > +--echo #Connection 3 > +--connect node_3, 127.0.0.1, root, , test, $NODE_MYPORT_3 > +--let $wait_condition = SELECT COUNT(*) = 1 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME= 't2'; > +--source include/wait_condition.inc > +--let $wait_condition = SELECT COUNT(*) = 6 FROM t2; > +--source include/wait_condition.inc > + > +INSERT INTO t2 VALUES(7,77); > +INSERT INTO t2 VALUES(8,88); > +SELECT @@global.gtid_binlog_state; > + > +#Perform SST > +--echo #Connection 1 > +--connection node_1 > +CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_before'); > +INSERT INTO t1 VALUES ('node1_committed_before'); > +COMMIT; > +--source include/save_master_gtid.inc > + > +--echo #Connection 2 > +--connection node_2 > +--source include/sync_with_master_gtid.inc > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node2_committed_before'); > +INSERT INTO t1 VALUES ('node2_committed_before'); > +COMMIT; > + > +--echo #Connection 3 > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 1 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME= 't1'; > +--source include/wait_condition.inc > +--let $wait_condition = SELECT COUNT(*) = 4 FROM t1; > +--source include/wait_condition.inc > +--let $node_1= node_2 > +--let $node_2= node_3 > +--source include/auto_increment_offset_save.inc > +--echo Shutting down server ... > +--source include/shutdown_mysqld.inc > + > + > +--echo #Connection 2 > +--connection node_2 > +--let $wait_condition = SELECT VARIABLE_VALUE = 1 FROM > INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = > 'wsrep_cluster_size' > +--source include/wait_condition.inc > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_during'); > +INSERT INTO t1 VALUES ('node1_committed_during'); > +COMMIT; > + > +--echo #Connection 3 > +--connection node_3 > +--echo Starting server ... > +--source include/start_mysqld.inc > +--source include/wait_until_ready.inc > +--source include/auto_increment_offset_restore.inc > + > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node3_committed_after'); > +INSERT INTO t1 VALUES ('node3_committed_after'); > +COMMIT; > + > +--echo #Connection 2 > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 8 FROM t1; > +--source include/wait_condition.inc > +Select * from t1 order by f1; > + > +--echo #Connection 3 > +--connection node_3 > +Select * from t1 order by f1; > + > +#SST Done > +--sleep 1 > +--echo #Connection 2 > +--connection node_2 > +SELECT @@global.gtid_binlog_state; > + > +--echo #Connection 3 > +--connection node_3 > +SELECT @@global.gtid_binlog_state; > + > +--echo #Connection 1 > +--connection node_1 > +SET AUTOCOMMIT=ON; > +#drop table t1; > +#CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > + > +--echo #Connection 2 > +--connection node_2 > +SET AUTOCOMMIT=ON; > +--echo #Connection 3 > +--connection node_3 > +SET AUTOCOMMIT=ON; > + > +# > +#stop slave on node 2 > +--echo #Connection 2 > +--connection node_2 > +STOP slave; > +--source include/wait_for_slave_to_stop.inc > +--sleep 1 > +INSERT INTO t1 VALUES ('node2_slave_stoped'); > + > +--echo #Connection 1 > +--connection node_1 > +INSERT INTO t1 VALUES ('node1_normal_entry'); > +--source include/save_master_gtid.inc > + > +#start slave > +--echo #Connection 2 > +--connection node_2 > +INSERT INTO t1 VALUES ('node2_slave_stoped_inserted'); > +start slave; > +--source include/wait_for_slave_to_start.inc > +--source include/sync_with_master_gtid.inc > +INSERT INTO t1 VALUES ('node2_slave_started'); > +SELECT count(*) from t1; > +SELECT @@global.gtid_binlog_state; > + > +--echo #Connection 3 > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 12 FROM t1; > +--source include/wait_condition.inc > +SELECT count(*) from t1; > +SELECT @@global.gtid_binlog_state; > + > +--echo #Connection 1 > +--connection node_1 > +DROP TABLE t2,t1; > + > +# Unfortunately without the sleep below the following statement fails > with "query returned no rows", which > +# is difficult to understand given that it is an aggregate query. A > "query execution was interrupted" > +# warning is also reported by MTR, which is also weird. > +# > + > +--sleep 3 > + > +--echo #Connection 2 > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't2'; > +--source include/wait_condition.inc > + > +--echo #Connection 3 > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--echo #Connection 2 > +--connection node_2 > +STOP SLAVE; > +RESET SLAVE; > +reset master; > + > +--echo #Connection 3 > +--connection node_3 > +reset master; > + > +--echo #Connection 1 > +--connection node_1 > +reset master; > + > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.cnf > b/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.cnf > new file mode 100644 > index 0000000..bb9c8e8 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.cnf > @@ -0,0 +1,18 @@ > +!include ../galera_2nodes_as_slave.cnf > + > +[mysqld] > +log-bin=mysqld-bin > +log-slave-updates > +binlog-format=ROW > +wsrep_sst_method=rsync > +[mysqld.1] > +gtid-domain-id=1 > +[mysqld.2] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > +[mysqld.3] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > + > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.test > b/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.test > new file mode 100644 > index 0000000..3fe94ad > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.test > @@ -0,0 +1,207 @@ > +# > +# Test Galera as a slave to a MariaDB master using GTIDs > +# > +# suite/galera/galera_2nodes_as_slave.cnf describes the setup of the > nodes > +# > +# In addition to performing DDL and DML, we check that the gtid of the > master is preserved inside the cluster > +# > + > +--source include/big_test.inc > +--source include/have_innodb.inc > +# As node #1 is not a Galera node, we connect to node #2 in order to run > include/galera_cluster.inc > +--connect node_2a, 127.0.0.1, root, , test, $NODE_MYPORT_2 > +--source include/galera_cluster.inc > + > +--echo #Connection 2 > +--connection node_2 > +--disable_query_log > +--eval CHANGE MASTER TO MASTER_HOST='127.0.0.1', MASTER_USER='root', > MASTER_PORT=$NODE_MYPORT_1,master_use_gtid=slave_pos; > +--enable_query_log > +START SLAVE; > +--sleep 1 > + > + > +--echo #Connection 1 > +--connection node_1 > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY, f2 int unique) ENGINE=InnoDB; > +INSERT INTO t2 VALUES(1,11); > +INSERT INTO t2 VALUES(2,22); > +INSERT INTO t2 VALUES(3,33); > + > +SELECT @@global.gtid_binlog_state; > +--source include/save_master_gtid.inc > + > +--echo #Connection 2 > +--connection node_2 > +--source include/sync_with_master_gtid.inc > +SELECT @@global.gtid_binlog_state; > + > +INSERT INTO t2 VALUES(4,44); > +INSERT INTO t2 VALUES(5,55); > +INSERT INTO t2 VALUES(6,66); > +SELECT @@global.gtid_binlog_state; > + > +--echo #Connection 3 > +--connect node_3, 127.0.0.1, root, , test, $NODE_MYPORT_3 > +--let $wait_condition = SELECT COUNT(*) = 1 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME= 't2'; > +--source include/wait_condition.inc > +--let $wait_condition = SELECT COUNT(*) = 6 FROM t2; > +--source include/wait_condition.inc > + > +INSERT INTO t2 VALUES(7,77); > +INSERT INTO t2 VALUES(8,88); > +SELECT @@global.gtid_binlog_state; > + > +#Perform SST > +--echo #Connection 1 > +--connection node_1 > +CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_before'); > +INSERT INTO t1 VALUES ('node1_committed_before'); > +COMMIT; > +--source include/save_master_gtid.inc > + > +--echo #Connection 2 > +--connection node_2 > +--source include/sync_with_master_gtid.inc > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node2_committed_before'); > +INSERT INTO t1 VALUES ('node2_committed_before'); > +COMMIT; > + > +--echo #Connection 3 > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 1 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME= 't1'; > +--source include/wait_condition.inc > +--let $wait_condition = SELECT COUNT(*) = 4 FROM t1; > +--source include/wait_condition.inc > +--let $node_1= node_2 > +--let $node_2= node_3 > +--source include/auto_increment_offset_save.inc > +--echo Shutting down server ... > +--source include/shutdown_mysqld.inc > + > + > +--echo #Connection 2 > +--connection node_2 > +--let $wait_condition = SELECT VARIABLE_VALUE = 1 FROM > INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = > 'wsrep_cluster_size' > +--source include/wait_condition.inc > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_during'); > +INSERT INTO t1 VALUES ('node1_committed_during'); > +COMMIT; > + > +--echo #Connection 3 > +--connection node_3 > +--echo Starting server ... > +--source include/start_mysqld.inc > +--source include/wait_until_ready.inc > +--source include/auto_increment_offset_restore.inc > + > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node3_committed_after'); > +INSERT INTO t1 VALUES ('node3_committed_after'); > +COMMIT; > + > +--echo #Connection 2 > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 8 FROM t1; > +--source include/wait_condition.inc > +Select * from t1 order by f1; > + > +--echo #Connection 3 > +--connection node_3 > +Select * from t1 order by f1; > + > +#SST Done > +--sleep 1 > +--echo #Connection 2 > +--connection node_2 > +SELECT @@global.gtid_binlog_state; > + > +--echo #Connection 3 > +--connection node_3 > +SELECT @@global.gtid_binlog_state; > + > +--echo #Connection 1 > +--connection node_1 > +SET AUTOCOMMIT=ON; > +#drop table t1; > +#CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > + > +--echo #Connection 2 > +--connection node_2 > +SET AUTOCOMMIT=ON; > +--echo #Connection 3 > +--connection node_3 > +SET AUTOCOMMIT=ON; > + > +# > +#stop slave on node 2 > +--echo #Connection 2 > +--connection node_2 > +STOP slave; > +--sleep 1 > +INSERT INTO t1 VALUES ('node2_slave_stoped'); > + > +--echo #Connection 1 > +--connection node_1 > +INSERT INTO t1 VALUES ('node1_normal_entry'); > +--source include/save_master_gtid.inc > + > +#start slave > +--echo #Connection 2 > +--connection node_2 > +INSERT INTO t1 VALUES ('node2_slave_stoped_inserted'); > +start slave; > +--source include/sync_with_master_gtid.inc > +INSERT INTO t1 VALUES ('node2_slave_started'); > +SELECT count(*) from t1; > +SELECT @@global.gtid_binlog_state; > + > +--echo #Connection 3 > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 12 FROM t1; > +--source include/wait_condition.inc > +SELECT count(*) from t1; > +SELECT @@global.gtid_binlog_state; > + > +--echo #Connection 1 > +--connection node_1 > +DROP TABLE t2,t1; > + > +# Unfortunately without the sleep below the following statement fails > with "query returned no rows", which > +# is difficult to understand given that it is an aggregate query. A > "query execution was interrupted" > +# warning is also reported by MTR, which is also weird. > +# > + > +--sleep 3 > + > +--echo #Connection 2 > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't2'; > +--source include/wait_condition.inc > + > +--echo #Connection 3 > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM > INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--echo #Connection 2 > +--connection node_2 > +STOP SLAVE; > +RESET SLAVE ALL; > +reset master; > + > +--echo #Connection 3 > +--connection node_3 > +reset master; > + > +--echo #Connection 1 > +--connection node_1 > +reset master; > diff --git a/mysql-test/suite/galera_3nodes/galera_2x3nodes.cnf > b/mysql-test/suite/galera_3nodes/galera_2x3nodes.cnf > new file mode 100644 > index 0000000..3f39b82 > --- /dev/null > +++ b/mysql-test/suite/galera_3nodes/galera_2x3nodes.cnf > @@ -0,0 +1,122 @@ > +# Use default setting for mysqld processes > +!include include/default_mysqld.cnf > + > +[mysqld] > +log-bin > +log-slave-updates > +binlog-format=row > +innodb-autoinc-lock-mode=2 > +default-storage-engine=innodb > +wsrep_gtid_mode=1 > +gtid_ignore_duplicates > + > +wsrep-on=1 > +wsrep-provider=(a)ENV.WSREP_PROVIDER > +wsrep_node_address=127.0.0.1 > +# enforce read-committed characteristics across the cluster > +# wsrep-causal-reads=ON > +# wsrep-sync-wait=15 > + > +[mysqld.1] > +#galera_port=(a)OPT.port > +#ist_port=(a)OPT.port > +#sst_port=(a)OPT.port > +wsrep-cluster-address='gcomm://' > +wsrep_provider_options='base_port=(a)mysqld.1.#galera_port; > evs.suspect_timeout=PT300S;evs.inactive_timeout=PT1000M; > evs.install_timeout=PT155S;evs.keepalive_period = PT100S' > + > +wsrep_sst_receive_address=127.0.0.2:@mysqld.1.#sst_port > +wsrep_node_incoming_address=127.0.0.1:@mysqld.1.port > +wsrep_sst_receive_address='127.0.0.1:@mysqld.1.#sst_port' > + > +[mysqld.2] > +#galera_port=(a)OPT.port > +#ist_port=(a)OPT.port > +#sst_port=(a)OPT.port > +wsrep_cluster_address='gcomm://127.0.0.1:@mysqld.1.#galera_port' > +wsrep_provider_options='base_port=(a)mysqld.2.#galera_port; > evs.suspect_timeout=PT300S;evs.inactive_timeout=PT1000M; > evs.install_timeout=PT155S;evs.keepalive_period = PT100S' > + > +wsrep_sst_receive_address=127.0.0.2:@mysqld.2.#sst_port > +wsrep_node_incoming_address=127.0.0.1:@mysqld.2.port > +wsrep_sst_receive_address='127.0.0.1:@mysqld.2.#sst_port' > + > +[mysqld.3] > +#galera_port=(a)OPT.port > +#ist_port=(a)OPT.port > +#sst_port=(a)OPT.port > +wsrep_cluster_address='gcomm://127.0.0.1:@mysqld.1.#galera_port' > +wsrep_provider_options='base_port=(a)mysqld.3.#galera_port; > evs.suspect_timeout=PT300S;evs.inactive_timeout=PT1000M; > evs.install_timeout=PT155S;evs.keepalive_period = PT100S' > + > +wsrep_sst_receive_address=127.0.0.2:@mysqld.3.#sst_port > +wsrep_node_incoming_address=127.0.0.1:@mysqld.3.port > +wsrep_sst_receive_address='127.0.0.1:@mysqld.3.#sst_port' > + > + > +[mysqld.4] > +wsrep_cluster_name=cluster2 > +#galera_port=(a)OPT.port > +#ist_port=(a)OPT.port > +#sst_port=(a)OPT.port > + > +wsrep-cluster-address='gcomm://' > +wsrep_provider_options='base_port=(a)mysqld.4.#galera_port; > evs.suspect_timeout=PT300S;evs.inactive_timeout=PT1000M; > evs.install_timeout=PT155S;evs.keepalive_period = PT100S' > + > +wsrep_sst_receive_address=127.0.0.2:@mysqld.4.#sst_port > +wsrep_node_incoming_address=127.0.0.1:@mysqld.4.port > +wsrep_sst_receive_address='127.0.0.1:@mysqld.4.#sst_port' > + > +[mysqld.5] > +wsrep_cluster_name=cluster2 > +#galera_port=(a)OPT.port > +#ist_port=(a)OPT.port > +#sst_port=(a)OPT.port > +wsrep_cluster_address='gcomm://127.0.0.1:@mysqld.4.#galera_port' > +wsrep_provider_options='base_port=(a)mysqld.5.#galera_port; > evs.suspect_timeout=PT300S;evs.inactive_timeout=PT1000M; > evs.install_timeout=PT155S;evs.keepalive_period = PT100S' > + > +wsrep_sst_receive_address=127.0.0.2:@mysqld.5.#sst_port > +wsrep_node_incoming_address=127.0.0.1:@mysqld.5.port > +wsrep_sst_receive_address='127.0.0.1:@mysqld.5.#sst_port' > + > +[mysqld.6] > +wsrep_cluster_name=cluster2 > +#galera_port=(a)OPT.port > +#ist_port=(a)OPT.port > +#sst_port=(a)OPT.port > +wsrep_cluster_address='gcomm://127.0.0.1:@mysqld.4.#galera_port' > +wsrep_provider_options='base_port=(a)mysqld.6.#galera_port; > evs.suspect_timeout=PT300S;evs.inactive_timeout=PT1000M; > evs.install_timeout=PT155S;evs.keepalive_period = PT100S' > + > +wsrep_sst_receive_address=127.0.0.2:@mysqld.6.#sst_port > +wsrep_node_incoming_address=127.0.0.1:@mysqld.6.port > +wsrep_sst_receive_address='127.0.0.1:@mysqld.6.#sst_port' > + > +[ENV] > +NODE_MYPORT_1= @mysqld.1.port > +NODE_MYSOCK_1= @mysqld.1.socket > + > +NODE_MYPORT_2= @mysqld.2.port > +NODE_MYSOCK_2= @mysqld.2.socket > + > +NODE_MYPORT_3= @mysqld.3.port > +NODE_MYSOCK_3= @mysqld.3.socket > + > +NODE_MYPORT_4= @mysqld.4.port > +NODE_MYSOCK_4= @mysqld.4.socket > + > +NODE_MYPORT_5= @mysqld.5.port > +NODE_MYSOCK_5= @mysqld.5.socket > + > +NODE_MYPORT_6= @mysqld.6.port > +NODE_MYSOCK_6= @mysqld.6.socket > + > +NODE_GALERAPORT_1= @mysqld.1.#galera_port > +NODE_GALERAPORT_2= @mysqld.2.#galera_port > +NODE_GALERAPORT_3= @mysqld.3.#galera_port > +NODE_GALERAPORT_4= @mysqld.4.#galera_port > +NODE_GALERAPORT_5= @mysqld.5.#galera_port > +NODE_GALERAPORT_6= @mysqld.6.#galera_port > + > +NODE_SSTPORT_1= @mysqld.1.#sst_port > +NODE_SSTPORT_2= @mysqld.2.#sst_port > +NODE_SSTPORT_3= @mysqld.3.#sst_port > +NODE_SSTPORT_4= @mysqld.4.#sst_port > +NODE_SSTPORT_5= @mysqld.5.#sst_port > +NODE_SSTPORT_6= @mysqld.6.#sst_port > diff --git a/mysql-test/suite/galera_3nodes/r/galera_gtid_2_cluster.result > b/mysql-test/suite/galera_3nodes/r/galera_gtid_2_cluster.result > new file mode 100644 > index 0000000..52c82ab > --- /dev/null > +++ b/mysql-test/suite/galera_3nodes/r/galera_gtid_2_cluster.result > @@ -0,0 +1,117 @@ > +cluster 1 node 1 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > +Variable_name Value > +wsrep_cluster_size 3 > +cluster 1 node 2 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > +Variable_name Value > +wsrep_cluster_size 3 > +cluster 1 node 3 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > +Variable_name Value > +wsrep_cluster_size 3 > +cluster 2 node 1 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > +Variable_name Value > +wsrep_cluster_size 3 > +cluster 2 node 2 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > +Variable_name Value > +wsrep_cluster_size 3 > +cluster 2 node 3 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > +Variable_name Value > +wsrep_cluster_size 3 > +change master to master_host='127.0.0.1', master_user='root', > master_port=16003, master_use_gtid=current_pos, ignore_server_ids=(12,13);; > +start slave; > +include/wait_for_slave_to_start.inc > +select @@gtid_binlog_state; > +@@gtid_binlog_state > + > +select @@gtid_slave_pos; > +@@gtid_slave_pos > + > +change master to master_host='127.0.0.1', master_user='root', > master_port=16000, master_use_gtid=current_pos, ignore_server_ids=(22,23);; > +start slave; > +include/wait_for_slave_to_start.inc > +select @@gtid_binlog_state; > +@@gtid_binlog_state > + > +select @@gtid_slave_pos; > +@@gtid_slave_pos > + > +cluster 1 node 1 > +create table t1 (cluster_domain_id int ,node_server_id int, seq_no int); > +insert into t1 values (1, 11, 2); > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-11-2 > +#wait for sync cluster 1 and 2 > +include/save_master_gtid.inc > +include/sync_with_master_gtid.inc > +cluster 2 node 1 > +insert into t1 values (2, 21, 1); > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-11-2,2-21-1 > +select * from t1; > +cluster_domain_id node_server_id seq_no > +1 11 2 > +2 21 1 > +#wait for sync cluster 1 and 2 > +include/save_master_gtid.inc > +include/sync_with_master_gtid.inc > +cluster 1 node 2 > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-11-2,2-21-1 > +insert into t1 values (1, 12, 3); > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-11-2,1-12-3,2-21-1 > +#wait for sync cluster 1 and 2 > +include/save_master_gtid.inc > +include/sync_with_master_gtid.inc > +cluster 1 node 3 > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-11-2,1-12-3,2-21-1 > +insert into t1 values (1, 13, 4); > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-12-3,1-11-2,1-13-4,2-21-1 > +#wait for sync cluster 1 and 2 > +include/save_master_gtid.inc > +include/sync_with_master_gtid.inc > +cluster 2 node 2 > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-12-3,1-11-2,1-13-4,2-21-1 > +insert into t1 values (2, 22, 2); > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-12-3,1-11-2,1-13-4,2-21-1,2-22-2 > +#wait for sync cluster 2 and 1 > +include/save_master_gtid.inc > +include/sync_with_master_gtid.inc > +cluster 2 node 3 > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-12-3,1-11-2,1-13-4,2-21-1,2-22-2 > +insert into t1 values (2, 23, 3); > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-12-3,1-11-2,1-13-4,2-21-1,2-22-2,2-23-3 > +#wait for sync cluster 2 and 1 > +include/save_master_gtid.inc > +include/sync_with_master_gtid.inc > +cluster 1 node 1 > +select @@gtid_binlog_state; > +@@gtid_binlog_state > +1-12-3,1-11-2,1-13-4,2-21-1,2-22-2,2-23-3 > +drop table t1; > +stop slave; > +change master to master_use_gtid=no, ignore_server_ids=(); > +cluster 2 node 1 > +stop slave; > +change master to master_use_gtid=no, ignore_server_ids=(); > diff --git a/mysql-test/suite/galera_3nodes/t/galera_gtid_2_cluster.cnf > b/mysql-test/suite/galera_3nodes/t/galera_gtid_2_cluster.cnf > new file mode 100644 > index 0000000..dc5535e > --- /dev/null > +++ b/mysql-test/suite/galera_3nodes/t/galera_gtid_2_cluster.cnf > @@ -0,0 +1,28 @@ > +# We need a dedicated .cnf file, even if empty, in order to force this > test to run > +# alone on a freshly started cluster. Otherwise there are adverse > interactions with > +# following tests such as galera_3nodes.galera_var_dirty_reads2 > + > +!include ../galera_2x3nodes.cnf > +[mysqld.1] > +wsrep_gtid_domain_id=1 > +server-id=11 > + > +[mysqld.2] > +wsrep_gtid_domain_id=1 > +server-id=12 > + > +[mysqld.3] > +wsrep_gtid_domain_id=1 > +server-id=13 > + > +[mysqld.4] > +wsrep_gtid_domain_id=2 > +server-id=21 > + > +[mysqld.5] > +wsrep_gtid_domain_id=2 > +server-id=22 > + > +[mysqld.6] > +wsrep_gtid_domain_id=2 > +server-id=23 > diff --git a/mysql-test/suite/galera_3nodes/t/galera_gtid_2_cluster.test > b/mysql-test/suite/galera_3nodes/t/galera_gtid_2_cluster.test > new file mode 100644 > index 0000000..4c9e611 > --- /dev/null > +++ b/mysql-test/suite/galera_3nodes/t/galera_gtid_2_cluster.test > @@ -0,0 +1,143 @@ > +# > +# This test creates 2x 3 nodes galera cluster. > +# The whole test case > +# A <-> B <-> C {Galera cluster 1} > +# | {Circular Async replication} > +# D <-> E <-> F {Galera cluster 2} > +# We will write on any random node to see if gtid is consitent or not > +# Then we will kill node D and set up the replication between A and E > +# To see whether fail over works or not. > + > +--source include/big_test.inc > +--source include/galera_cluster.inc > +--source include/have_innodb.inc > + > +--connection node_1 > +--echo cluster 1 node 1 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > + > +--connection node_2 > +--echo cluster 1 node 2 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > + > +--connect node_3, 127.0.0.1, root, , test, $NODE_MYPORT_3 > +--connection node_3 > +--echo cluster 1 node 3 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > + > +--connect node_4, 127.0.0.1, root, , test, $NODE_MYPORT_4 > +--connection node_4 > +--echo cluster 2 node 1 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > + > +--connect node_5, 127.0.0.1, root, , test, $NODE_MYPORT_5 > +--connection node_5 > +--echo cluster 2 node 2 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > + > +--connect node_6, 127.0.0.1, root, , test, $NODE_MYPORT_6 > +--connection node_6 > +--echo cluster 2 node 3 > +SHOW STATUS LIKE 'wsrep_cluster_size'; > + > +--connection node_1 > +--eval change master to master_host='127.0.0.1', master_user='root', > master_port=$NODE_MYPORT_4, master_use_gtid=current_pos, > ignore_server_ids=(12,13); > +start slave; > +--source include/wait_for_slave_to_start.inc > +select @@gtid_binlog_state; > +select @@gtid_slave_pos; > +#--query_vertical SHOW SLAVE STATUS; > + > +--connection node_4 > +--eval change master to master_host='127.0.0.1', master_user='root', > master_port=$NODE_MYPORT_1, master_use_gtid=current_pos, > ignore_server_ids=(22,23); > +start slave; > +--source include/wait_for_slave_to_start.inc > +select @@gtid_binlog_state; > +select @@gtid_slave_pos; > +#--query_vertical SHOW SLAVE STATUS; > + > +--echo cluster 1 node 1 > +--connection node_1 > +create table t1 (cluster_domain_id int ,node_server_id int, seq_no int); > +insert into t1 values (1, 11, 2); > +select @@gtid_binlog_state; > + > +--echo #wait for sync cluster 1 and 2 > +--connection node_1 > +--source include/save_master_gtid.inc > +--connection node_4 > +--source include/sync_with_master_gtid.inc > + > +--echo cluster 2 node 1 > +--connection node_4 > +insert into t1 values (2, 21, 1); > +select @@gtid_binlog_state; > +select * from t1; > + > +--echo #wait for sync cluster 1 and 2 > +--connection node_1 > +--source include/save_master_gtid.inc > +--connection node_4 > +--source include/sync_with_master_gtid.inc > + > + > +--echo cluster 1 node 2 > +--connection node_2 > +select @@gtid_binlog_state; > +insert into t1 values (1, 12, 3); > +select @@gtid_binlog_state; > + > +--echo #wait for sync cluster 1 and 2 > +--connection node_1 > +--source include/save_master_gtid.inc > +--connection node_4 > +--source include/sync_with_master_gtid.inc > + > +--echo cluster 1 node 3 > +--connection node_3 > +select @@gtid_binlog_state; > +insert into t1 values (1, 13, 4); > +select @@gtid_binlog_state; > + > +--echo #wait for sync cluster 1 and 2 > +--connection node_1 > +--source include/save_master_gtid.inc > +--connection node_4 > +--source include/sync_with_master_gtid.inc > + > +--echo cluster 2 node 2 > +--connection node_5 > +select @@gtid_binlog_state; > +insert into t1 values (2, 22, 2); > +select @@gtid_binlog_state; > + > +--echo #wait for sync cluster 2 and 1 > +--connection node_4 > +--source include/save_master_gtid.inc > +--connection node_1 > +--source include/sync_with_master_gtid.inc > + > +--echo cluster 2 node 3 > +--connection node_6 > +select @@gtid_binlog_state; > +insert into t1 values (2, 23, 3); > +select @@gtid_binlog_state; > + > +--echo #wait for sync cluster 2 and 1 > +--connection node_4 > +--source include/save_master_gtid.inc > +--connection node_1 > +--source include/sync_with_master_gtid.inc > + > + > +--echo cluster 1 node 1 > +--connection node_1 > +select @@gtid_binlog_state; > +drop table t1; > +stop slave; > +change master to master_use_gtid=no, ignore_server_ids=(); > + > +--echo cluster 2 node 1 > +--connection node_4 > +stop slave; > +change master to master_use_gtid=no, ignore_server_ids=(); > diff --git a/sql/log.cc b/sql/log.cc > index 07d4074..bb8b1d3 100644 > --- a/sql/log.cc > +++ b/sql/log.cc > @@ -5544,7 +5544,36 @@ THD::binlog_start_trans_and_stmt() > cache_mngr->trx_cache.get_prev_position() == MY_OFF_T_UNDEF) > { > this->binlog_set_stmt_begin(); > - if (in_multi_stmt_transaction_mode()) > + bool mstmt_mode= in_multi_stmt_transaction_mode(); > +#ifdef WITH_WSREP > + /* Write Gtid > + Get domain id only when gtid mode is set > + If this event is replicate through a master then , > + we will forward the same gtid another nodes > + We have to do this only one time in mysql transaction. > + Since this function is called multiple times , We will check for > + ha_info->is_started() > + */ > + Ha_trx_info *ha_info; > + ha_info= this->ha_data[binlog_hton->slot].ha_info + (mstmt_mode ? 1 > : 0); > + > + if (!ha_info->is_started() && wsrep_gtid_mode && > this->variables.gtid_seq_no) > + { > + binlog_cache_mngr *const cache_mngr= > + (binlog_cache_mngr*) thd_get_ha_data(this, binlog_hton); > + > + IO_CACHE *file= > + cache_mngr->get_binlog_cache_log(use_trans_cache(this, true)); > + Log_event_writer writer(file); > + Gtid_log_event gtid_event(this, this->variables.gtid_seq_no, > + this->variables.gtid_domain_id, > + true, LOG_EVENT_SUPPRESS_USE_F, > + true, 0); > + gtid_event.server_id= this->variables.server_id; > + writer.write(&gtid_event); > + } > +#endif > + if (mstmt_mode) > trans_register_ha(this, TRUE, binlog_hton); > trans_register_ha(this, FALSE, binlog_hton); > /* > @@ -5826,7 +5855,7 @@ MYSQL_BIN_LOG::write_gtid_event(THD *thd, bool > standalone, > DBUG_PRINT("enter", ("standalone: %d", standalone)); > > #ifdef WITH_WSREP > - if (WSREP(thd) && thd->wsrep_trx_meta.gtid.seqno != -1 && > wsrep_gtid_mode) > + if (WSREP(thd) && thd->wsrep_trx_meta.gtid.seqno != -1 && > wsrep_gtid_mode && !thd->variables.gtid_seq_no) > { > domain_id= wsrep_gtid_domain_id; > } else { > @@ -6840,6 +6869,24 @@ int MYSQL_BIN_LOG::write_cache(THD *thd, IO_CACHE > *cache) > mysql_mutex_assert_owner(&LOCK_log); > if (reinit_io_cache(cache, READ_CACHE, 0, 0, 0)) > DBUG_RETURN(ER_ERROR_ON_WRITE); > + uchar *read_pos= cache->read_pos; > + > +#ifdef WITH_WSREP > + /* > + In the cache we have gtid event if , below condition is true, > + then we will simply increment the read position of cache by > + the length of Gtid_log_event. > + */ > + if (wsrep_gtid_mode && thd->variables.gtid_seq_no) > + { > + uchar *head= read_pos; > + uint data_len= uint4korr(head + EVENT_LEN_OFFSET); > + uint event_type= (uchar)head[EVENT_TYPE_OFFSET]; > + if (event_type == GTID_LOG_EVENT) > + cache->read_pos+= data_len; > + } > +#endif > + > uint length= my_b_bytes_in_cache(cache), group, carry, hdr_offs; > long val; > ulong end_log_pos_inc= 0; // each event processed adds > BINLOG_CHECKSUM_LEN 2 t > diff --git a/sql/wsrep_mysqld.cc b/sql/wsrep_mysqld.cc > index bdc8491..285bb52 100644 > --- a/sql/wsrep_mysqld.cc > +++ b/sql/wsrep_mysqld.cc > @@ -1227,6 +1227,16 @@ int wsrep_to_buf_helper( > if (!ret && writer.write(&gtid_ev)) ret= 1; > } > #endif /* GTID_SUPPORT */ > + if (wsrep_gtid_mode && thd->variables.gtid_seq_no) > + { > + Gtid_log_event gtid_event(thd, thd->variables.gtid_seq_no, > + thd->variables.gtid_domain_id, > + true, LOG_EVENT_SUPPRESS_USE_F, > + true, 0); > + gtid_event.server_id= thd->variables.server_id; > + if (!gtid_event.is_valid()) ret= 0; > + ret= writer.write(&gtid_event); > + } > > /* if there is prepare query, add event for it */ > if (!ret && thd->wsrep_TOI_pre_query) > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] Issue with commit 0f43279c / MDEV-13936
by jerome brauge 12 Dec '17

12 Dec '17

Hi All, Since some time ago, my app crashed the server (10.2.-ext) in some case (repeatable) with the following stack trace : On Linux Centos 7: ./sql/mysqld(my_print_stacktrace+0x38)[0x7f463f49a093] ./sql/mysqld(handle_fatal_signal+0x3a3)[0x7f463ed2747a] /lib64/libpthread.so.0(+0xf370)[0x7f463dffd370] ./sql/mysqld(_ZN13Explain_query10get_selectEj+0x40)[0x7f463ebf4d06] ./sql/mysqld(_ZN13Explain_query8add_nodeEP12Explain_node+0x203)[0x7f463ebf4f15] ./sql/mysqld(_ZN4JOIN24save_explain_data_internEP13Explain_querybbbPKc+0x959)[0x7f463eaf3db9] ./sql/mysqld(_ZN4JOIN17save_explain_dataEP13Explain_querybbbb+0x14b)[0x7f463eabb963] ./sql/mysqld(_ZN4JOIN13build_explainEv+0xe2)[0x7f463eab3b70] ./sql/mysqld(_ZN4JOIN8optimizeEv+0xb6)[0x7f463eab3d2e] ./sql/mysqld(_ZN13st_select_lex31optimize_unflattened_subqueriesEb+0x263)[0x7f463ea57b5d] ./sql/mysqld(_ZN4JOIN31optimize_unflattened_subqueriesEv+0x24)[0x7f463ec1b746] ./sql/mysqld(_ZN4JOIN15optimize_stage2Ev+0x1cdb)[0x7f463eab7171] ./sql/mysqld(_ZN4JOIN14optimize_innerEv+0x14dd)[0x7f463eab542d] ./sql/mysqld(_ZN4JOIN8optimizeEv+0x7a)[0x7f463eab3cf2] ./sql/mysqld(_Z12mysql_selectP3THDP10TABLE_LISTjR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x263)[0x7f463eabccd2] ./sql/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x16c)[0x7f463eab13e6] ./sql/mysqld(_Z21mysql_execute_commandP3THD+0x4b5b)[0x7f463ea76fb4] ./sql/mysqld(_ZN13sp_instr_stmt9exec_coreEP3THDPj+0x80)[0x7f463eeaafb0] ./sql/mysqld(_ZN13sp_lex_keeper23reset_lex_and_exec_coreEP3THDPjbP8sp_instr+0x1d1)[0x7f463eeaa549] ./sql/mysqld(_ZN13sp_instr_stmt7executeEP3THDPj+0x1b5)[0x7f463eeaabf9] ./sql/mysqld(_ZN7sp_head7executeEP3THDb+0x754)[0x7f463eea4ff6] ./sql/mysqld(_ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE+0x971)[0x7f463eea74df] ./sql/mysqld(+0x645959)[0x7f463ea71959] ./sql/mysqld(_ZN12Sql_cmd_call7executeEP3THD+0x270)[0x7f463ea7240c] ./sql/mysqld(_Z21mysql_execute_commandP3THD+0x9f95)[0x7f463ea7c3ee] ./sql/mysqld(_ZN18Prepared_statement7executeEP6Stringb+0x508)[0x7f463ea9e468] ./sql/mysqld(_ZN18Prepared_statement12execute_loopEP6StringbPhS2_+0x200)[0x7f463ea9c6fa] ./sql/mysqld(+0x66e113)[0x7f463ea9a113] ./sql/mysqld(_Z19mysqld_stmt_executeP3THDPcj+0x97)[0x7f463ea99cd7] ./sql/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0xc9e)[0x7f463ea6e68d] ./sql/mysqld(_Z10do_commandP3THD+0x700)[0x7f463ea6d315] ./sql/mysqld(_Z24do_handle_one_connectionP7CONNECT+0x1ca)[0x7f463ebbe3d2] ./sql/mysqld(handle_one_connection+0x30)[0x7f463ebbe125] ./sql/mysqld(+0x1012852)[0x7f463f43e852] /lib64/libpthread.so.0(+0x7dc5)[0x7f463dff5dc5] /lib64/libc.so.6(clone+0x6d)[0x7f463c01673d] Or on Windows : Unhandled exception thrown: read access violation. Dynamic_array<Explain_select *>::at(...) returned 0xDB925000. > mysqld.exe!Explain_query::~Explain_query() Line 71 C++ mysqld.exe!Explain_query::`scalar deleting destructor'(unsigned int) C++ mysqld.exe!delete_explain_query(LEX * lex) Line 2397 C++ mysqld.exe!sp_lex_keeper::reset_lex_and_exec_core(THD * thd, unsigned int * nextp, bool open_tables, sp_instr * instr) Line 3130 C++ mysqld.exe!sp_instr_stmt::execute(THD * thd, unsigned int * nextp) Line 3266 C++ mysqld.exe!sp_head::execute(THD * thd, bool merge_da_on_success) Line 1213 C++ mysqld.exe!sp_head::execute_procedure(THD * thd, List<Item> * args) Line 2072 C++ mysqld.exe!do_execute_sp(THD * thd, sp_head * sp) Line 2907 C++ mysqld.exe!Sql_cmd_call::execute(THD * thd) Line 3147 C++ mysqld.exe!mysql_execute_command(THD * thd) Line 6223 C++ mysqld.exe!Prepared_statement::execute(String * expanded_query, bool open_cursor) Line 4682 C++ mysqld.exe!Prepared_statement::execute_loop(String * expanded_query, bool open_cursor, unsigned char * packet, unsigned char * packet_end) Line 4111 C++ mysqld.exe!mysql_stmt_execute_common(THD * thd, unsigned long stmt_id, unsigned char * packet, unsigned char * packet_end, unsigned long cursor_flags, bool bulk_op, bool read_types) Line 3132 C++ mysqld.exe!mysqld_stmt_execute(THD * thd, char * packet_arg, unsigned int packet_length) Line 3028 C++ mysqld.exe!dispatch_command(enum_server_command command, THD * thd, char * packet, unsigned int packet_length, bool is_com_multi, bool is_next_command) Line 1763 C++ mysqld.exe!do_command(THD * thd) Line 1369 C++ mysqld.exe!threadpool_process_request(THD * thd) Line 366 C++ mysqld.exe!tp_callback(TP_connection * c) Line 192 C++ mysqld.exe!tp_callback(_TP_CALLBACK_INSTANCE * instance, void * context) Line 377 C++ mysqld.exe!work_callback(_TP_CALLBACK_INSTANCE * instance, void * context, _TP_WORK * work) Line 451 C++ The sql statement involved (in a stored procedure): Query (0x7f457df65bb8): INSERT INTO qnwtc( enrqnwtc, priqnwtc, lprqnwtc, cprqnwtc, nuiqnwtc) SELECT 'PRI' ,occgtpar ,intgtpar ,ch1gtpar , NAME_CONST('wpio_ident',110325)FROM gtpara WHERE padgtpar = 'PRIQNTAC' AND numgtets = NAME_CONST('wsysets',_utf8'IFR' COLLATE 'utf8_bin')AND etagtpar = 'A' Where gtpara is a view. If the stored procedure is called alone, it works fine. I've revert commit 0f43279c (MDEV-13936) and now my app works fine again. Regards.

1 0

Re: [Maria-developers] Mdev-10715 Initial Patch
by Sachin Setiya 11 Dec '17

11 Dec '17

Hi Andrei! Thanks for the review. On Thu, Oct 12, 2017 at 11:48 PM, <andrei.elkin(a)pp.inet.fi> wrote: > Howdy, Sachin! > > I've checked the patch mostly from the high-level design pov to gather > few suggestions. Find them on the top of diff hunks (hopefully the diff > remains applicable) below indented with '*review*>'. > > > > revision-id: daf9a67ea13c5c7309f85a104e7c07ec68f65e88 (mariadb-10.1.28-10-gdaf9a67) > parent(s): 5eb666ad37ff80e5f8dc714bb68fcb1e269b797a > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2017-10-10 17:45:21 +0530 > message: > > Mdev-10715 Initial Patch > *review*> The initial patch is a very good time to write down the work > intent and its key features. Done > Could you please do that and highlight how WSREP has really > become multi-source compatible. > > > --- > mysql-test/suite/galera/r/galera_gtid_slave.result | 18 +++ > .../r/galera_gtid_slave_sst_mysqldump.result | 92 ++++++++++++ > .../galera/r/galera_gtid_slave_sst_rsync.result | 92 ++++++++++++ > mysql-test/suite/galera/t/galera_gtid_slave.cnf | 18 +++ > mysql-test/suite/galera/t/galera_gtid_slave.test | 67 +++++++++ > .../galera/t/galera_gtid_slave_sst_mysqldump.cnf | 18 +++ > .../galera/t/galera_gtid_slave_sst_mysqldump.test | 156 +++++++++++++++++++++ > .../suite/galera/t/galera_gtid_slave_sst_rsync.cnf | 19 +++ > .../galera/t/galera_gtid_slave_sst_rsync.test | 156 +++++++++++++++++++++ > sql/log.cc | 37 ++++- > sql/wsrep_mysqld.cc | 9 ++ > 11 files changed, 681 insertions(+), 1 deletion(-) > > diff --git a/mysql-test/suite/galera/r/galera_gtid_slave.result b/mysql-test/suite/galera/r/galera_gtid_slave.result > new file mode 100644 > index 0000000..7ab10c7 > --- /dev/null > +++ b/mysql-test/suite/galera/r/galera_gtid_slave.result > @@ -0,0 +1,18 @@ > +START SLAVE; > +CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB; > +INSERT INTO t1 VALUES(1); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-2 > +INSERT INTO t1 VALUES(2); > +INSERT INTO t1 VALUES(3); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-2,2-2-2 > +INSERT INTO t1 VALUES(4); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-2,2-2-2,2-3-3 > +DROP TABLE t1; > +STOP SLAVE; > +RESET SLAVE ALL; > diff --git a/mysql-test/suite/galera/r/galera_gtid_slave_sst_mysqldump.result b/mysql-test/suite/galera/r/galera_gtid_slave_sst_mysqldump.result > new file mode 100644 > index 0000000..8e8e2e0 > --- /dev/null > +++ b/mysql-test/suite/galera/r/galera_gtid_slave_sst_mysqldump.result > @@ -0,0 +1,92 @@ > +START SLAVE; > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY, f2 int unique) ENGINE=InnoDB; > +INSERT INTO t2 VALUES(1,11); > +INSERT INTO t2 VALUES(2,22); > +INSERT INTO t2 VALUES(3,33); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4 > +INSERT INTO t2 VALUES(4,44); > +INSERT INTO t2 VALUES(5,55); > +INSERT INTO t2 VALUES(6,66); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-3 > +INSERT INTO t2 VALUES(7,77); > +INSERT INTO t2 VALUES(8,88); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-3,2-3-5 > +CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_before'); > +INSERT INTO t1 VALUES ('node1_committed_before'); > +COMMIT; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node2_committed_before'); > +INSERT INTO t1 VALUES ('node2_committed_before'); > +COMMIT; > +Shutting down server ... > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_during'); > +INSERT INTO t1 VALUES ('node1_committed_during'); > +COMMIT; > +Starting server ... > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node3_committed_after'); > +INSERT INTO t1 VALUES ('node3_committed_after'); > +COMMIT; > +Select * from t1 order by f1; > +f1 > +node1_committed_before > +node1_committed_before > +node1_committed_during > +node1_committed_during > +node2_committed_before > +node2_committed_before > +node3_committed_after > +node3_committed_after > +Select * from t1 order by f1; > +f1 > +node1_committed_before > +node1_committed_before > +node1_committed_during > +node1_committed_during > +node2_committed_before > +node2_committed_before > +node3_committed_after > +node3_committed_after > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-2-7,2-3-8 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-2-7,2-3-8 > +SET AUTOCOMMIT=ON; > +SET AUTOCOMMIT=ON; > +SET AUTOCOMMIT=ON; > +STOP slave; > +INSERT INTO t1 VALUES ('node2_slave_stoped'); > +INSERT INTO t1 VALUES ('node1_slave_stoped'); > +INSERT INTO t1 VALUES ('node2_slave_stoped_inserted'); > +start slave; > +INSERT INTO t1 VALUES ('node2_slave_started'); > +SELECT COUNT(*) from t1; > +COUNT(*) > +11 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-3-8,2-2-11 > +SELECT COUNT(*) from t1; > +COUNT(*) > +11 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-3-8,2-2-11 > +DROP TABLE t2,t1; > +STOP SLAVE; > +RESET SLAVE ALL; > diff --git a/mysql-test/suite/galera/r/galera_gtid_slave_sst_rsync.result b/mysql-test/suite/galera/r/galera_gtid_slave_sst_rsync.result > new file mode 100644 > index 0000000..9ab4601 > --- /dev/null > +++ b/mysql-test/suite/galera/r/galera_gtid_slave_sst_rsync.result > @@ -0,0 +1,92 @@ > +START SLAVE; > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY, f2 int unique) ENGINE=InnoDB; > +INSERT INTO t2 VALUES(1,11); > +INSERT INTO t2 VALUES(2,22); > +INSERT INTO t2 VALUES(3,33); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4 > +INSERT INTO t2 VALUES(4,44); > +INSERT INTO t2 VALUES(5,55); > +INSERT INTO t2 VALUES(6,66); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-3 > +INSERT INTO t2 VALUES(7,77); > +INSERT INTO t2 VALUES(8,88); > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-4,2-2-3,2-3-5 > +CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_before'); > +INSERT INTO t1 VALUES ('node1_committed_before'); > +COMMIT; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node2_committed_before'); > +INSERT INTO t1 VALUES ('node2_committed_before'); > +COMMIT; > +Shutting down server ... > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_during'); > +INSERT INTO t1 VALUES ('node1_committed_during'); > +COMMIT; > +Starting server ... > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node3_committed_after'); > +INSERT INTO t1 VALUES ('node3_committed_after'); > +COMMIT; > +Select * from t1 order by f1; > +f1 > +node1_committed_before > +node1_committed_before > +node1_committed_during > +node1_committed_during > +node2_committed_before > +node2_committed_before > +node3_committed_after > +node3_committed_after > +Select * from t1 order by f1; > +f1 > +node1_committed_before > +node1_committed_before > +node1_committed_during > +node1_committed_during > +node2_committed_before > +node2_committed_before > +node3_committed_after > +node3_committed_after > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-2-7,2-3-8 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-6,2-2-7,2-3-8 > +SET AUTOCOMMIT=ON; > +SET AUTOCOMMIT=ON; > +SET AUTOCOMMIT=ON; > +STOP slave; > +INSERT INTO t1 VALUES ('node2_slave_stoped'); > +INSERT INTO t1 VALUES ('node1_slave_stoped'); > +INSERT INTO t1 VALUES ('node2_slave_stoped_inserted'); > +start slave; > +INSERT INTO t1 VALUES ('node2_slave_started'); > +SELECT COUNT(*) from t1; > +COUNT(*) > +11 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-7,2-3-8,2-2-11 > +SELECT COUNT(*) from t1; > +COUNT(*) > +12 > +SELECT @@global.gtid_binlog_state; > +@@global.gtid_binlog_state > +1-1-7,2-3-8,2-2-11 > +DROP TABLE t2,t1; > +STOP SLAVE; > +RESET SLAVE ALL; > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave.cnf b/mysql-test/suite/galera/t/galera_gtid_slave.cnf > new file mode 100644 > index 0000000..409d0d1 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave.cnf > @@ -0,0 +1,18 @@ > +!include ../galera_2nodes_as_slave.cnf > + > +[mysqld] > +log-bin=mysqld-bin > +log-slave-updates > +binlog-format=ROW > + > +[mysqld.1] > +gtid-domain-id=1 > +[mysqld.2] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > +[mysqld.3] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > + > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave.test b/mysql-test/suite/galera/t/galera_gtid_slave.test > new file mode 100644 > index 0000000..3b3f6f3 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave.test > @@ -0,0 +1,67 @@ > +# > +# Test Galera as a slave to a MariaDB master using GTIDs > +# > +# suite/galera/galera_2nodes_as_slave.cnf describes the setup of the nodes > +# suite/galera/t/galera_as_slave_gtid.cnf has the GTID options > +# > +# In addition to performing DDL and DML, we check that the gtid of the master is preserved inside the cluster > +# > + > +--source include/have_innodb.inc > + > +# As node #1 is not a Galera node, we connect to node #2 in order to run include/galera_cluster.inc > +--connect node_2a, 127.0.0.1, root, , test, $NODE_MYPORT_2 > +--source include/galera_cluster.inc > + > +--connection node_2 > +--disable_query_log > +--eval CHANGE MASTER TO MASTER_HOST='127.0.0.1', MASTER_USER='root', MASTER_PORT=$NODE_MYPORT_1; > +--enable_query_log > +START SLAVE; > + > +--connection node_1 > +CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB; > +INSERT INTO t1 VALUES(1); > + > +SELECT @@global.gtid_binlog_state; > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 1 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--let $wait_condition = SELECT COUNT(*) = 1 FROM t1; > +--source include/wait_condition.inc > +INSERT INTO t1 VALUES(2); > +INSERT INTO t1 VALUES(3); > +SELECT @@global.gtid_binlog_state; > + > +--connect node_3, 127.0.0.1, root, , test, $NODE_MYPORT_3 > +--sleep 1 > +--let $wait_condition = SELECT COUNT(*) = 3 FROM t1; > +--source include/wait_condition.inc > + > +INSERT INTO t1 VALUES(4); > +SELECT @@global.gtid_binlog_state; > + > +--connection node_1 > +DROP TABLE t1; > + > +# > +# Unfortunately without the sleep below the following statement fails with "query returned no rows", which > +# is difficult to understand given that it is an aggregate query. A "query execution was interrupted" > +# warning is also reported by MTR, which is also weird. > +# > + > +--sleep 1 > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--connection node_2 > +STOP SLAVE; > +RESET SLAVE ALL; > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.cnf b/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.cnf > new file mode 100644 > index 0000000..409d0d1 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.cnf > @@ -0,0 +1,18 @@ > +!include ../galera_2nodes_as_slave.cnf > + > +[mysqld] > +log-bin=mysqld-bin > +log-slave-updates > +binlog-format=ROW > + > +[mysqld.1] > +gtid-domain-id=1 > +[mysqld.2] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > +[mysqld.3] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > + > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.test b/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.test > new file mode 100644 > index 0000000..67aad21 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave_sst_mysqldump.test > @@ -0,0 +1,156 @@ > +# > +# Test Galera as a slave to a MariaDB master using GTIDs > +# > +# suite/galera/galera_2nodes_as_slave.cnf describes the setup of the nodes > +# suite/galera/t/galera_as_slave_gtid.cnf has the GTID options > +# > +# In addition to performing DDL and DML, we check that the gtid of the master is preserved inside the cluster > +# > + > +--source include/big_test.inc > +--source include/have_innodb.inc > +# As node #1 is not a Galera node, we connect to node #2 in order to run include/galera_cluster.inc > +--connect node_2a, 127.0.0.1, root, , test, $NODE_MYPORT_2 > +--source include/galera_cluster.inc > + > +--connection node_2 > +--disable_query_log > +--eval CHANGE MASTER TO MASTER_HOST='127.0.0.1', MASTER_USER='root', MASTER_PORT=$NODE_MYPORT_1; > +--enable_query_log > +START SLAVE; > + > +--connection node_1 > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY, f2 int unique) ENGINE=InnoDB; > +INSERT INTO t2 VALUES(1,11); > +INSERT INTO t2 VALUES(2,22); > +INSERT INTO t2 VALUES(3,33); > + > +SELECT @@global.gtid_binlog_state; > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 1 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't2'; > +--source include/wait_condition.inc > + > +--let $wait_condition = SELECT COUNT(*) = 3 FROM t2; > +--source include/wait_condition.inc > +INSERT INTO t2 VALUES(4,44); > +INSERT INTO t2 VALUES(5,55); > +INSERT INTO t2 VALUES(6,66); > +SELECT @@global.gtid_binlog_state; > + > +--connect node_3, 127.0.0.1, root, , test, $NODE_MYPORT_3 > +--sleep 1 > +--let $wait_condition = SELECT COUNT(*) = 6 FROM t2; > +--source include/wait_condition.inc > + > +INSERT INTO t2 VALUES(7,77); > +INSERT INTO t2 VALUES(8,88); > +SELECT @@global.gtid_binlog_state; > + > +#Perform SST > +--connection node_1 > +CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_before'); > +INSERT INTO t1 VALUES ('node1_committed_before'); > +COMMIT; > + > +--connection node_2 > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node2_committed_before'); > +INSERT INTO t1 VALUES ('node2_committed_before'); > +COMMIT; > + > +--connection node_3 > +--echo Shutting down server ... > +--source include/shutdown_mysqld.inc > + > + > +--connection node_2 > +--let $wait_condition = SELECT VARIABLE_VALUE = 1 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size' > +--source include/wait_condition.inc > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_during'); > +INSERT INTO t1 VALUES ('node1_committed_during'); > +COMMIT; > + > +--connection node_3 > +--echo Starting server ... > +--source include/start_mysqld.inc > +--source include/wait_until_ready.inc > + > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node3_committed_after'); > +INSERT INTO t1 VALUES ('node3_committed_after'); > +COMMIT; > + > +--connection node_2 > +Select * from t1 order by f1; > + > +--connection node_3 > +Select * from t1 order by f1; > + > +#SST Done > +--sleep 1 > +--connection node_2 > +SELECT @@global.gtid_binlog_state; > + > +--connection node_3 > +SELECT @@global.gtid_binlog_state; > + > +# > +#stop slave on node 2 > +--connection node_1 > +SET AUTOCOMMIT=ON; > +#drop table t1; > +#CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > + > +--connection node_2 > +SET AUTOCOMMIT=ON; > +--connection node_3 > +SET AUTOCOMMIT=ON; > + > +--connection node_2 > +STOP slave; > +--sleep 1 > +INSERT INTO t1 VALUES ('node2_slave_stoped'); > + > +--connection node_1 > +INSERT INTO t1 VALUES ('node1_slave_stoped'); > +#start slave > +--connection node_2 > +INSERT INTO t1 VALUES ('node2_slave_stoped_inserted'); > +start slave; > +INSERT INTO t1 VALUES ('node2_slave_started'); > +SELECT COUNT(*) from t1; > +SELECT @@global.gtid_binlog_state; > + > +--connection node_3 > +SELECT COUNT(*) from t1; > +SELECT @@global.gtid_binlog_state; > + > +--connection node_1 > +DROP TABLE t2,t1; > + > +# Unfortunately without the sleep below the following statement fails with "query returned no rows", which > +# is difficult to understand given that it is an aggregate query. A "query execution was interrupted" > +# warning is also reported by MTR, which is also weird. > +# > + > +--sleep 3 > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't2'; > +--source include/wait_condition.inc > + > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--connection node_2 > +STOP SLAVE; > +RESET SLAVE ALL; > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.cnf b/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.cnf > new file mode 100644 > index 0000000..2c82531 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.cnf > @@ -0,0 +1,19 @@ > +!include ../galera_2nodes_as_slave.cnf > + > +[mysqld] > +log-bin=mysqld-bin > +log-slave-updates > +binlog-format=ROW > +wsrep_sst_method=rsync > + > +[mysqld.1] > +gtid-domain-id=1 > +[mysqld.2] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > +[mysqld.3] > +gtid-domain-id=2 > +wsrep_gtid_mode=1 > +wsrep_gtid_domain_id=2 > + > diff --git a/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.test b/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.test > new file mode 100644 > index 0000000..67aad21 > --- /dev/null > +++ b/mysql-test/suite/galera/t/galera_gtid_slave_sst_rsync.test > @@ -0,0 +1,156 @@ > +# > +# Test Galera as a slave to a MariaDB master using GTIDs > +# > +# suite/galera/galera_2nodes_as_slave.cnf describes the setup of the nodes > +# suite/galera/t/galera_as_slave_gtid.cnf has the GTID options > +# > +# In addition to performing DDL and DML, we check that the gtid of the master is preserved inside the cluster > +# > + > +--source include/big_test.inc > +--source include/have_innodb.inc > +# As node #1 is not a Galera node, we connect to node #2 in order to run include/galera_cluster.inc > +--connect node_2a, 127.0.0.1, root, , test, $NODE_MYPORT_2 > +--source include/galera_cluster.inc > + > +--connection node_2 > +--disable_query_log > +--eval CHANGE MASTER TO MASTER_HOST='127.0.0.1', MASTER_USER='root', MASTER_PORT=$NODE_MYPORT_1; > +--enable_query_log > +START SLAVE; > + > +--connection node_1 > +CREATE TABLE t2 (f1 INTEGER PRIMARY KEY, f2 int unique) ENGINE=InnoDB; > +INSERT INTO t2 VALUES(1,11); > +INSERT INTO t2 VALUES(2,22); > +INSERT INTO t2 VALUES(3,33); > + > +SELECT @@global.gtid_binlog_state; > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 1 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't2'; > +--source include/wait_condition.inc > + > +--let $wait_condition = SELECT COUNT(*) = 3 FROM t2; > +--source include/wait_condition.inc > +INSERT INTO t2 VALUES(4,44); > +INSERT INTO t2 VALUES(5,55); > +INSERT INTO t2 VALUES(6,66); > +SELECT @@global.gtid_binlog_state; > + > +--connect node_3, 127.0.0.1, root, , test, $NODE_MYPORT_3 > +--sleep 1 > +--let $wait_condition = SELECT COUNT(*) = 6 FROM t2; > +--source include/wait_condition.inc > + > +INSERT INTO t2 VALUES(7,77); > +INSERT INTO t2 VALUES(8,88); > +SELECT @@global.gtid_binlog_state; > + > +#Perform SST > +--connection node_1 > +CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_before'); > +INSERT INTO t1 VALUES ('node1_committed_before'); > +COMMIT; > + > +--connection node_2 > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node2_committed_before'); > +INSERT INTO t1 VALUES ('node2_committed_before'); > +COMMIT; > + > +--connection node_3 > +--echo Shutting down server ... > +--source include/shutdown_mysqld.inc > + > + > +--connection node_2 > +--let $wait_condition = SELECT VARIABLE_VALUE = 1 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size' > +--source include/wait_condition.inc > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node1_committed_during'); > +INSERT INTO t1 VALUES ('node1_committed_during'); > +COMMIT; > + > +--connection node_3 > +--echo Starting server ... > +--source include/start_mysqld.inc > +--source include/wait_until_ready.inc > + > +SET AUTOCOMMIT=OFF; > +START TRANSACTION; > +INSERT INTO t1 VALUES ('node3_committed_after'); > +INSERT INTO t1 VALUES ('node3_committed_after'); > +COMMIT; > + > +--connection node_2 > +Select * from t1 order by f1; > + > +--connection node_3 > +Select * from t1 order by f1; > + > +#SST Done > +--sleep 1 > +--connection node_2 > +SELECT @@global.gtid_binlog_state; > + > +--connection node_3 > +SELECT @@global.gtid_binlog_state; > + > +# > +#stop slave on node 2 > +--connection node_1 > +SET AUTOCOMMIT=ON; > +#drop table t1; > +#CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; > + > +--connection node_2 > +SET AUTOCOMMIT=ON; > +--connection node_3 > +SET AUTOCOMMIT=ON; > + > +--connection node_2 > +STOP slave; > +--sleep 1 > +INSERT INTO t1 VALUES ('node2_slave_stoped'); > + > +--connection node_1 > +INSERT INTO t1 VALUES ('node1_slave_stoped'); > +#start slave > +--connection node_2 > +INSERT INTO t1 VALUES ('node2_slave_stoped_inserted'); > +start slave; > +INSERT INTO t1 VALUES ('node2_slave_started'); > +SELECT COUNT(*) from t1; > +SELECT @@global.gtid_binlog_state; > + > +--connection node_3 > +SELECT COUNT(*) from t1; > +SELECT @@global.gtid_binlog_state; > + > +--connection node_1 > +DROP TABLE t2,t1; > + > +# Unfortunately without the sleep below the following statement fails with "query returned no rows", which > +# is difficult to understand given that it is an aggregate query. A "query execution was interrupted" > +# warning is also reported by MTR, which is also weird. > +# > + > +--sleep 3 > + > +--connection node_2 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't2'; > +--source include/wait_condition.inc > + > +--connection node_3 > +--let $wait_condition = SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1'; > +--source include/wait_condition.inc > + > +--connection node_2 > +STOP SLAVE; > +RESET SLAVE ALL; > > *review*> The better timing for the non-zero seq_no Gtid write deems to > inside THD::binlog_start_trans_and_stmt() invoked few instructions > earlier. This method is invoked once per *transaction* which > guarantees no duplication which the current chosen point seems to > concede. Perhaps you change its singature to pass the writer. Done as we discussed. this is added into binlog_statrt_trans_and_stmt() And I am checking for ha_info->is_started(). > Also I think we may write the non-zero seq_no Gtid generally, regardless > of WITH_WSREP (naturally to block doing that later at binlog commit > phase, also commented below). This commit will go into 10.1 so as we discussed , we can do this later in 10.3. > > diff --git a/sql/log.cc b/sql/log.cc > index a9f486d..53eeadf 100644 > --- a/sql/log.cc > +++ b/sql/log.cc > @@ -5655,6 +5655,23 @@ int THD::binlog_write_table_map(TABLE *table, bool is_transactional, > DBUG_RETURN(error); > } > } > +#ifdef WITH_WSREP > + /* Get domain id only when gtid mode is set > + If this event is replicate through a master then , > + we will forward the same gtid another nodes > + */ > + if (wsrep_gtid_mode && this->variables.gtid_seq_no) > + { > + Gtid_log_event gtid_event(this, this->variables.gtid_seq_no, > + this->variables.gtid_domain_id, > + true, LOG_EVENT_SUPPRESS_USE_F, > + true, 0); > + gtid_event.server_id= this->variables.server_id; > + > + if ((error= writer.write(&gtid_event))) > + DBUG_RETURN(error); > + } > +#endif > if ((error= writer.write(&the_event))) > DBUG_RETURN(error); > > @@ -5826,7 +5843,7 @@ MYSQL_BIN_LOG::write_gtid_event(THD *thd, bool standalone, > DBUG_PRINT("enter", ("standalone: %d", standalone)); > > #ifdef WITH_WSREP > - if (WSREP(thd) && thd->wsrep_trx_meta.gtid.seqno != -1 && wsrep_gtid_mode) > + if (WSREP(thd) && thd->wsrep_trx_meta.gtid.seqno != -1 && wsrep_gtid_mode && !thd->variables.gtid_seq_no) > { > domain_id= wsrep_gtid_domain_id; > } else { > > *review*> I'm suggest we check the non-zero seq_no status in > MYSQL_BIN_LOG::write_gtid_event() > > *review*>diff --git a/sql/log.cc b/sql/log.cc > *review*>index 2fcb6f6dbf6..8229426a5bd 100644 > *review*>--- a/sql/log.cc > *review*>+++ b/sql/log.cc > *review*>@@ -5867,7 +5867,6 @@ MYSQL_BIN_LOG::write_gtid_event(THD *thd, bool standalone, > *review*> err= rpl_global_gtid_binlog_state.update_with_next_gtid(domain_id, > *review*> local_server_id, &gtid); > *review*> seq_no= gtid.seq_no; > *review*>- } > *review*> if (err) > *review*> DBUG_RETURN(true); > *review*> thd->last_commit_gtid= gtid; > *review*>@@ -5880,6 +5879,7 @@ MYSQL_BIN_LOG::write_gtid_event(THD *thd, bool standalone, > *review*> DBUG_ASSERT(this == &mysql_bin_log); > *review*> if (write_event(&gtid_event)) > *review*> DBUG_RETURN(true); > *review*>+ } > *review*> status_var_add(thd->status_var.binlog_bytes_written, gtid_event.data_written); > *review*> > *review*> DBUG_RETURN(false); > > *review*> so only the zero auto-evaluated seq_no Gtid would make to the > method you're trying to patch. And the hunk is unneeded in this > case. Like above we generalize WITH_WSREP away. > ^^^^^^^ I am not sure about this hunk , I think this was mistake. I have reverted this change. > @@ -6718,6 +6735,24 @@ int MYSQL_BIN_LOG::write_cache(THD *thd, IO_CACHE *cache) > mysql_mutex_assert_owner(&LOCK_log); > if (reinit_io_cache(cache, READ_CACHE, 0, 0, 0)) > DBUG_RETURN(ER_ERROR_ON_WRITE); > + uchar *read_pos= cache->read_pos; > + > +#ifdef WITH_WSREP > + /* > + In the cache we have gtid event if , below condition is true, > + then we will simply increment the read position of cache by > + the length of Gtid_log_event. > + */ > + if (wsrep_gtid_mode && thd->variables.gtid_seq_no) > + { > + uchar *head= read_pos; > + uint data_len= uint4korr(head + EVENT_LEN_OFFSET); > + uint event_type= (uchar)head[EVENT_TYPE_OFFSET]; > + if (event_type == GTID_LOG_EVENT) > + cache->read_pos+= data_len; > + } > +#endif > + > uint length= my_b_bytes_in_cache(cache), group, carry, hdr_offs; > long val; > ulong end_log_pos_inc= 0; // each event processed adds BINLOG_CHECKSUM_LEN 2 t > > *review*> The following block is fine *but* we shall explain > what is GTID_SUPPORT, whether it actually for us. > Well , there is no such thing as GTID_SUPPORT :( , we are using ifdef without even defining gtid_support ╭─sachin@sachin-desk ~/10715/server ‹bb-10.1-10715*› ╰─$ grep -Rnie 'GTID_SUPPORT' sql/wsrep_sst.cc:290:#ifdef GTID_SUPPORT sql/wsrep_sst.cc:292:#endif /* GTID_SUPPORT */ sql/wsrep_applier.cc:130:#ifdef GTID_SUPPORT sql/wsrep_applier.cc:141:#endif /* GTID_SUPPORT */ sql/wsrep_applier.cc:309:#ifdef GTID_SUPPORT sql/wsrep_applier.cc:311:#endif /* GTID_SUPPORT */ sql/wsrep_mysqld.cc:50:#ifdef GTID_SUPPORT sql/wsrep_mysqld.cc:53:#endif /* GTID_SUPPORT */ sql/wsrep_mysqld.cc:256:#ifdef GTID_SUPPORT sql/wsrep_mysqld.cc:274:#endif /* GTID_SUPPORT */ sql/wsrep_mysqld.cc:413:#ifdef GTID_SUPPORT sql/wsrep_mysqld.cc:415:#endif /* GTID_SUPPORT */ sql/wsrep_mysqld.cc:1222:#ifdef GTID_SUPPORT sql/wsrep_mysqld.cc:1229:#endif /* GTID_SUPPORT */ sql/sql_class.h:4085:#ifdef GTID_SUPPORT sql/sql_class.h:4087:#endif /* GTID_SUPPORT */ sql/wsrep_mysqld.h:261:#ifdef GTID_SUPPORT sql/wsrep_mysqld.h:263:#endif /* GTID_SUPPORT */ > diff --git a/sql/wsrep_mysqld.cc b/sql/wsrep_mysqld.cc > index 33689a9..65751f1 100644 > --- a/sql/wsrep_mysqld.cc > +++ b/sql/wsrep_mysqld.cc > @@ -1227,6 +1227,15 @@ int wsrep_to_buf_helper( > if (!ret && writer.write(&gtid_ev)) ret= 1; > } > #endif /* GTID_SUPPORT */ > + if (wsrep_gtid_mode && thd->variables.gtid_seq_no) > + { > + Gtid_log_event gtid_event(thd, thd->variables.gtid_seq_no, > + thd->variables.gtid_domain_id, > + true, LOG_EVENT_SUPPRESS_USE_F, > + true, 0); > + gtid_event.server_id= thd->variables.server_id; > + ret= writer.write(&gtid_event); > + } > > /* if there is prepare query, add event for it */ > if (!ret && thd->wsrep_TOI_pre_query) -- Regards Sachin Setiya Software Engineer at MariaDB

1 0

[Maria-developers] Mysterious files after build
by Michal Schorm 09 Dec '17

09 Dec '17

Hello, I'd like to know more about some files, that are left after MariaDB build in Fedora. I do not ship them, however I haven't found any explanation from previous maintainers. I gathered some clues and thoughts about them. Questions are: What are those files and what they does?; Is it safe to ship them?; Should I start to ship them? /usr/bin/mysql_embedded - you (MariaDB) ship this file in client package %{_datadir}/mariadb/binary-configure - this script creates the MySQL system tables and starts the server. %{_datadir}/mariadb/magic - FS files first-bytes recoginiton? %{_datadir}/mariadb/mysql.server %{_datadir}/%{pkg_name}/mysqld_multi.server - those shouldn't be needed on systemd OSs ? /usr/bin/mytop - you (mariaDB) do not ship this file -- Michal Schorm Associate Software Engineer Core Services - Databases Team Red Hat

2 1

Re: [Maria-developers] b610986: MDEV-14593 human-readable XA RECOVER
by Sergei Golubchik 08 Dec '17

08 Dec '17

Hi, Alexey! Please, see few comments below. On Dec 07, Alexey Botchkov wrote: > revision-id: b610986c9c226e9b9cbe7c80ceaead1de0a55535 (mariadb-10.2.11-26-gb610986) > parent(s): 4d016e6ed2d32298977a66e125cbfcae39e23847 > committer: Alexey Botchkov > timestamp: 2017-12-07 23:49:12 +0400 > message: > > MDEV-14593 human-readable XA RECOVER. > > diff --git a/sql/handler.cc b/sql/handler.cc > index 7eed722..128721c 100644 > --- a/sql/handler.cc > +++ b/sql/handler.cc > @@ -1978,6 +1978,97 @@ int ha_recover(HASH *commit_list) > } > > /** > + return the XID as it appears in the SQL function's arguments. > + So this string can be passed to XA START, XA PREPARE etc... > + > + @note > + the 'buf' has to have space for at least SQL_XIDSIZE bytes. > +*/ > + > + > +/* > + 'a'..'z' 'A'..'Z', '0'..'9' > + and '-' '_' ' ' symbols don't have to be > + converted. I'd say it's overcomplication, why not to convert everything to hex? > +*/ > + > +static const char xid_needs_conv[128]= > +{ > + 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, > + 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, > + 0,1,1,1,1,1,1,1,1,1,1,1,1,0,1,1, > + 0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1, > + 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > + 0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,0, > + 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > + 0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1 > +}; > + > +uint get_sql_xid(XID *xid, char *buf) > +{ > + int tot_len= xid->gtrid_length + xid->bqual_length; > + int i; > + const char *orig_buf= buf; > + > + for (i=0; i<tot_len; i++) > + { > + uchar c= ((uchar *) xid->data)[i] > + if (c >= 128 || xid_needs_conv[c]) > + break; > + } > + > + if (i >= tot_len) > + { > + /* No need to convert characters to hexadecimals. */ > + *buf++= '\''; > + memcpy(buf, xid->data, xid->gtrid_length); > + buf+= xid->gtrid_length; > + *buf++= '\''; > + if (xid->bqual_length > 0 || xid->formatID != 1) > + { > + *buf++= ','; > + *buf++= '\''; > + memcpy(buf, xid->data+xid->gtrid_length, xid->bqual_length); > + buf+= xid->bqual_length; > + *buf++= '\''; > + } > + } > + else > + { > + *buf++= 'X'; > + *buf++= '\''; > + for (i= 0; i < xid->gtrid_length; i++) > + { > + *buf++=_dig_vec_lower[((uchar*) xid->data)[i] >> 4]; > + *buf++=_dig_vec_lower[((uchar*) xid->data)[i] & 0x0f]; > + } > + *buf++= '\''; > + if (xid->bqual_length > 0 || xid->formatID != 1) > + { > + *buf++= ','; > + *buf++= 'X'; > + *buf++= '\''; > + for (; i < tot_len; i++) > + { > + *buf++=_dig_vec_lower[((uchar*) xid->data)[i] >> 4]; > + *buf++=_dig_vec_lower[((uchar*) xid->data)[i] & 0x0f]; > + } > + *buf++= '\''; > + } > + } > + > + if (xid->formatID != 1) > + { > + *buf++= ','; > + buf+= my_longlong10_to_str_8bit(&my_charset_bin, buf, > + MY_INT64_NUM_DECIMAL_DIGITS, -10, xid->formatID); > + } > + > + return buf - orig_buf; > +} > + > + > +/** > return the list of XID's to a client, the same way SHOW commands do. > > @note > diff --git a/sql/handler.h b/sql/handler.h > index 486ba56..91e3168 100644 > --- a/sql/handler.h > +++ b/sql/handler.h > @@ -628,6 +628,16 @@ struct xid_t { > }; > typedef struct xid_t XID; > > +/* > + The size of XID string representation in the form > + 'gtrid', 'bqual', formatID > + see xid_t::get_sql_string() for details. > +*/ > +#define SQL_XIDSIZE (XIDDATASIZE * 2 + 8 + MY_INT64_NUM_DECIMAL_DIGITS) > +/* The 'buf' has to have space for at least SQL_XIDSIZE bytes. */ > +uint get_sql_xid(XID *xid, char *buf); Not sure you need this as an extern function in the header. Why not to keep it static in handler.cc? > /* for recover() handlerton call */ > #define MIN_XID_LIST_SIZE 128 > #define MAX_XID_LIST_SIZE (1024*128) > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index 17e6904..d0fa642 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -1705,6 +1705,7 @@ bool my_yyoverflow(short **a, YYSTYPE **b, ulong *yystacksize); > %token WRITE_SYM /* SQL-2003-N */ > %token X509_SYM > %token XA_SYM > +%token XID_SYM > %token XML_SYM > %token XOR > %token YEAR_MONTH_SYM You forgot to add XID_SYM to the keyword_sp rule. Also I don't see why we should have CONVERT XID syntax if the output is incompatible. Perhaps it'd be better to have a different syntax? FORMAT=SQL? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] Crash on view with union
by jerome brauge 08 Dec '17

08 Dec '17

Hi all, The following script crashs the server (built with last bb-10.2-ext sources). use test; CREATE or replace TABLE `gtfmt` ( `numgtfmt` char(10) COLLATE utf8_bin NOT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin; create or replace view gtfmtl(numgtfmt) as select 'x' from gtfmt union select 'x' from gtfmt ; SELECT * FROM gtfmtl WHERE numgtfmt = NAME_CONST('wnumgtfmt',_utf8'QEDITIONS' COLLATE 'utf8_bin'); Stack : > mysqld.exe!my_sigabrt_handler(int sig) Line 488 C mysqld.exe!raise(int signum) Line 516 C++ mysqld.exe!abort() Line 64 C++ mysqld.exe!common_assert_to_stderr_direct(const wchar_t * const expression, const wchar_t * const file_name, const unsigned int line_number) Line 123 C++ mysqld.exe!common_assert_to_stderr<wchar_t>(const wchar_t * const expression, const wchar_t * const file_name, const unsigned int line_number) Line 142 C++ mysqld.exe!_wassert(const wchar_t * expression, const wchar_t * file_name, unsigned int line_number) Line 404 C++ mysqld.exe!Item_func::fix_fields(THD * thd, Item * * ref) Line 330 C++ mysqld.exe!Item_str_func::fix_fields(THD * thd, Item * * ref) Line 106 C++ mysqld.exe!Item_name_const::fix_fields(THD * thd, Item * * ref) Line 2186 C++ mysqld.exe!Item_func::fix_fields(THD * thd, Item * * ref) Line 361 C++ mysqld.exe!JOIN::optimize_inner() Line 1311 C++ mysqld.exe!JOIN::optimize() Line 1121 C++ mysqld.exe!st_select_lex_unit::optimize() Line 1344 C++ mysqld.exe!mysql_derived_optimize(THD * thd, LEX * lex, TABLE_LIST * derived) Line 887 C++ mysqld.exe!mysql_handle_single_derived(LEX * lex, TABLE_LIST * derived, unsigned int phases) Line 197 C++ mysqld.exe!JOIN::optimize_inner() Line 1356 C++ mysqld.exe!JOIN::optimize() Line 1121 C++ mysqld.exe!mysql_select(THD * thd, TABLE_LIST * tables, unsigned int wild_num, List<Item> & fields, Item * conds, unsigned int og_num, st_order * order, st_order * group, Item * having, st_order * proc_param, unsigned __int64 select_options, select_result * result, st_select_lex_unit * unit, st_select_lex * select_lex) Line 3758 C++ mysqld.exe!handle_select(THD * thd, LEX * lex, select_result * result, unsigned long setup_tables_done_option) Line 364 C++ mysqld.exe!execute_sqlcom_select(THD * thd, TABLE_LIST * all_tables) Line 6473 C++ mysqld.exe!mysql_execute_command(THD * thd) Line 3717 C++ mysqld.exe!mysql_parse(THD * thd, char * rawbuf, unsigned int length, Parser_state * parser_state, bool is_com_multi, bool is_next_command) Line 7935 C++ mysqld.exe!dispatch_command(enum_server_command command, THD * thd, char * packet, unsigned int packet_length, bool is_com_multi, bool is_next_command) Line 1820 C++ mysqld.exe!do_command(THD * thd) Line 1369 C++ mysqld.exe!threadpool_process_request(THD * thd) Line 366 C++ mysqld.exe!tp_callback(TP_connection * c) Line 192 C++ mysqld.exe!tp_callback(_TP_CALLBACK_INSTANCE * instance, void * context) Line 377 C++ mysqld.exe!work_callback(_TP_CALLBACK_INSTANCE * instance, void * context, _TP_WORK * work) Line 451 C++ Best regard, Jérôme.

1 0

[Maria-developers] -Wnon-virtual-dtor and scripts in
by Alexander Barkov 08 Dec '17

08 Dec '17

Hello Vlad, Are we still supposed to use these scripts in the BUILD directory: compile-pentium64-debug compile-pentium64-debug-all compile-pentium64-debug-max compile-pentium64-gcov compile-pentium64-gprof compile-pentium64-max compile-pentium64-valgrind-max I stopped using them long time ago. But these scripts set some extra warnings for gcc, so now it's easy to overlook some warnings. Monty is still using these scripts and gets those extra warnings. In particular, -Wnon-virtual-dtor is often a subject of warnings. Is there a way to enable the -Wnon-virtual-dtor for gcc in the default build, i.e. when just using "cmake" instead of BUILD scripts? Thanks!

2 2

Re: [Maria-developers] 78a1779: mysqldump fix for invisible column
by Sergei Golubchik 07 Dec '17

07 Dec '17

Hi, Sachin! Looks fine. Two comments: 1. optimization idea, see below 2. add a test case that actually restores from the dump (and checks that it was restored correctly). mysqldump.test has examples, search for $MYSQL. After that - please push! On Dec 07, sachin wrote: > revision-id: 78a1779f29ed8d2a96658583065270de8bdd14b4 (mariadb-10.3.2-61-g78a1779) > parent(s): e8c8300a1c7e7e2b99e98c5c6986df69943cb661 > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2017-12-07 16:09:16 +0530 > message: > > mysqldump fix for invisible column > > Actually there are 2 issues in the case of invisible columns > > 1st `select fields from t1` will have more fields then `select * from t1`. > So instead of `select * from t1` we are using `select a,b,invisible from t1` > these fields are supplied from `select fields from t1`. > > 2nd We are using --complete-insert when we detect that this table is using > invisible columns. > > diff --git a/client/mysqldump.c b/client/mysqldump.c > index a260065..818bc07 100644 > --- a/client/mysqldump.c > +++ b/client/mysqldump.c > @@ -2971,6 +2979,12 @@ static uint get_table_structure(char *table, char *db, char *table_type, > DBUG_RETURN(0); > } > > + while ((row= mysql_fetch_row(result))) > + { > + if (strlen(row[SHOW_EXTRA]) && strstr(row[SHOW_EXTRA],"INVISIBLE")) > + complete_insert= 1; > + } > + mysql_data_seek(result, 0); > /* > If write_data is true, then we build up insert statements for > the table's data. Note: in subsequent lines of code, this test > @@ -3006,10 +3020,16 @@ static uint get_table_structure(char *table, char *db, char *table_type, > { > dynstr_append_checked(&insert_pat, ", "); > } > - init=1; > dynstr_append_checked(&insert_pat, > quote_name(row[SHOW_FIELDNAME], name_buff, 0)); > } > + if (init) > + { > + dynstr_append_checked(&select_field_names, ", "); > + } > + dynstr_append_checked(&select_field_names, > + quote_name(row[SHOW_FIELDNAME], name_buff, 0)); > + init=1; I think, you can do it with just one loop: while ((row= mysql_fetch_row(result))) { if (strlen(row[SHOW_EXTRA]) && strstr(row[SHOW_EXTRA],"INVISIBLE")) complete_insert= 1; if (init) dynstr_append_checked(&select_field_names, ", "); dynstr_append_checked(&select_field_names, quote_name(row[SHOW_FIELDNAME], name_buff, 0)); init=1; } if (complete_insert) dynstr_append_checked(&insert_pat, select_field_names->str); > } > num_fields= mysql_num_rows(result); > mysql_free_result(result); Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 339dd3d: Invisible Column mysqldump bug fix
by Sergei Golubchik 07 Dec '17

07 Dec '17

Hi, Sachin! Summary: 1. Add a test case, it should include a mix of tables with and without invisible columns, and columns with strange names (reserved words, spaces). 2. Always use full SELECT syntax, even if the table has no invisible columns. On Dec 05, sachin wrote: > revision-id: 339dd3d7508a1e5c7c014b5a59ed99ff0ad96542 (mariadb-10.3.2-59-g339dd3d) > parent(s): 6ed3374c8a67ae64448783ab107d1a11cb87f40c > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2017-12-05 05:35:28 +0530 > message: > > Invisible Column mysqldump bug fix > > --- > client/mysqldump.c | 46 +++++++++++++++++++++++++++++++++++++++++----- > 1 file changed, 41 insertions(+), 5 deletions(-) > > diff --git a/client/mysqldump.c b/client/mysqldump.c > index a260065..e59b203 100644 > --- a/client/mysqldump.c > +++ b/client/mysqldump.c > @@ -115,7 +115,7 @@ static my_bool verbose= 0, opt_no_create_info= 0, opt_no_data= 0, opt_no_data_m > opt_events= 0, opt_comments_used= 0, > opt_alltspcs=0, opt_notspcs= 0, opt_logging, > opt_drop_trigger= 0 ; > -static my_bool insert_pat_inited= 0, debug_info_flag= 0, debug_check_flag= 0; > +static my_bool insert_pat_inited= 0, debug_info_flag= 0, debug_check_flag= 0, invisble_table= 0; typo, *invisible* > static ulong opt_max_allowed_packet, opt_net_buffer_length; > static MYSQL mysql_connection,*mysql=0; > static DYNAMIC_STRING insert_pat; > @@ -2726,7 +2726,7 @@ static uint get_table_structure(char *table, char *db, char *table_type, > complete_insert= 0; > if ((write_data= !(*ignore_flag & IGNORE_DATA))) > { > - complete_insert= opt_complete_insert; > + invisble_table= complete_insert= opt_complete_insert; > if (!insert_pat_inited) > { > insert_pat_inited= 1; > @@ -2971,6 +2971,12 @@ static uint get_table_structure(char *table, char *db, char *table_type, > DBUG_RETURN(0); > } > > + while ((row= mysql_fetch_row(result))) > + { > + if (strlen(row[SHOW_EXTRA]) && strstr(row[SHOW_EXTRA],"INVISIBLE")) > + invisble_table= complete_insert= opt_complete_insert= 1; I think, you need to set invisible_table flag per table, just as you do now, but you should not change complete_insert or opt_complete_insert. That is, if a database has many tables, some with invisible columns and some have only normal columns - tables with invisible columns should use complete_insert, while other tables should not. Basically, INSERT should contain column names if (invisible_table || opt_complete_insert) > + } > + mysql_data_seek(result, 0); > /* > If write_data is true, then we build up insert statements for > the table's data. Note: in subsequent lines of code, this test > @@ -3617,7 +3623,7 @@ static void dump_table(char *table, char *db) > { > char ignore_flag; > char buf[200], table_buff[NAME_LEN+3]; > - DYNAMIC_STRING query_string; > + DYNAMIC_STRING query_string, select_field_names; > char table_type[NAME_LEN]; > char *result_table, table_buff2[NAME_LEN*2+3], *opt_quoted_table; > int error= 0; > @@ -3687,7 +3693,25 @@ static void dump_table(char *table, char *db) > verbose_msg("-- Sending SELECT query...\n"); > > init_dynamic_string_checked(&query_string, "", 1024, 1024); > + init_dynamic_string_checked(&select_field_names, "", 1024, 1024); > > + if (invisble_table) > + { > + /* > + Copy field name from insert_pat into select_field_names because > + select * from table will leave hidden columns. > + We will search for "(" in insert_pat and will copy the whole data > + into select_field_names , and then we will search for ")" in > + select_field_names and decrese the lenght accordingly. > + */ > + char *ptr= insert_pat.str; > + while ( *ptr++ != '(' ){} > + dynstr_append_checked(&select_field_names, ptr); > + ptr= &select_field_names.str[select_field_names.length - 1]; > + while ( *ptr-- != ')') > + select_field_names.length--; > + select_field_names.str[select_field_names.length- 1]= 0; please, check how INSERT code is doing it, you need to quote column names. > + } > if (path) > { > char filename[FN_REFLEN], tmp_path[FN_REFLEN]; > @@ -3708,7 +3732,13 @@ static void dump_table(char *table, char *db) > > /* now build the query string */ > > - dynstr_append_checked(&query_string, "SELECT /*!40001 SQL_NO_CACHE */ * INTO OUTFILE '"); > + dynstr_append_checked(&query_string, "SELECT /*!40001 SQL_NO_CACHE */ "); > + if (invisble_table) > + dynstr_append_checked(&query_string, select_field_names.str); > + else > + dynstr_append_checked(&query_string, "*"); I don't think you need to bother, you can always have a complete column list here. It should not break anything. > + dynstr_append_checked(&query_string, " INTO OUTFILE '"); > + dynstr_free(&select_field_names); > dynstr_append_checked(&query_string, filename); > dynstr_append_checked(&query_string, "'"); > > @@ -3757,7 +3787,13 @@ static void dump_table(char *table, char *db) > "\n--\n-- Dumping data for table %s\n--\n", > fix_for_comment(result_table)); > > - dynstr_append_checked(&query_string, "SELECT /*!40001 SQL_NO_CACHE */ * FROM "); > + dynstr_append_checked(&query_string, "SELECT /*!40001 SQL_NO_CACHE */ "); > + if (invisble_table) > + dynstr_append_checked(&query_string, select_field_names.str); > + else > + dynstr_append_checked(&query_string, "*"); same > + dynstr_free(&select_field_names); > + dynstr_append_checked(&query_string, " FROM "); > dynstr_append_checked(&query_string, result_table); > > if (where) > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 1

Re: [Maria-developers] 0f47171: MDEV-14005 Remove need for Partition Key to be part of Primary Key.
by Sergei Golubchik 06 Dec '17

06 Dec '17

Hi, Alexey! On Dec 04, Alexey Botchkov wrote: > revision-id: 0f4717118e135c178fc1b12065e33ac8a5e21d0b (mariadb-10.3.2-82-g0f47171) > parent(s): ddac2d7a1ef725361c7e83b819cb0bec72db024a > committer: Alexey Botchkov > timestamp: 2017-12-04 17:45:13 +0400 > message: > > MDEV-14005 Remove need for Partition Key to be part of Primary Key. > > Now we check all partitions for the unique keys if necessary > in ha_partition::write_row and ha_partition::update_row. > > diff --git a/sql/ha_partition.cc b/sql/ha_partition.cc > index 6bebf5d..12a6287 100644 > --- a/sql/ha_partition.cc > +++ b/sql/ha_partition.cc > @@ -372,6 +372,7 @@ void ha_partition::init_handler_variables() > part_share= NULL; > m_new_partitions_share_refs.empty(); > m_part_ids_sorted_by_num_of_records= NULL; > + m_cu_errkey= (uint) -1; > > #ifdef DONT_HAVE_TO_BE_INITALIZED > m_start_key.flag= 0; > @@ -4088,6 +4089,79 @@ void ha_partition::try_semi_consistent_read(bool yes) > } > > > +int ha_partition::check_files_for_key(uchar *key, int n_key, > + int partition_to_skip, int *res) > +{ > +// uint num_subparts= part_info->num_subparts; > + for (uint n=0; n < m_part_info->num_parts; n++) > + { > + handler *f= m_file[n]; > + int ires; > + > + if ((int) n == partition_to_skip) > + continue; > + > + f->active_index= n_key; > + if ((*res= f->extra(HA_EXTRA_KEYREAD)) || > + (*res= f->ha_index_init(n_key, FALSE))) > + return 1; > + > + *res= f->index_read_map(m_part_info->table->record[0], > + key, HA_WHOLE_KEY, HA_READ_KEY_EXACT); use index_read_idx_map here > + ires= f->ha_index_end(); > + > + if (*res == HA_ERR_KEY_NOT_FOUND) > + continue; > + > + if (*res == 0) > + *res= ires ? ires : HA_ERR_FOUND_DUPP_KEY; > + > + m_last_part= n; > + m_cu_errkey= n_key; > + return 1; > + } > + > + *res= 0; > + return 0; > +} > + > + > +int ha_partition::check_uniques_insert(uchar *buf, > + int partition_to_skip, int *res) > +{ > + uint n_key; > + > + for (n_key=0; n_key < m_part_info->n_uniques_to_check; n_key++) > + { can you use key_copy() here? > + uchar *cbuf= m_part_info->unique_key_buf[0]; > + KEY *ckey= m_part_info->uniques_to_check[n_key]; > + uint n; > + > + for (n=0; n < ckey->user_defined_key_parts; n++) > + { > + const KEY_PART_INFO *cpart= ckey->key_part + n; > + uint maybe_null= MY_TEST(cpart->null_bit); > + Field *f= cpart->field; > + my_ptrdiff_t ofs= buf-table->record[0]; > + > + f->move_field_offset(ofs); > + if (maybe_null) > + cbuf[0]= f->is_null(); > + f->get_key_image(cbuf+maybe_null, cpart->length, Field::itRAW); > + cbuf+= cpart->store_length; > + f->move_field_offset(-ofs); > + } > + > + if (check_files_for_key(m_part_info->unique_key_buf[0], n_key, > + partition_to_skip, res)) > + return 1; > + } > + > + *res= 0; > + return 0; > +} > + > + > /**************************************************************************** > MODULE change record > ****************************************************************************/ > @@ -4199,6 +4273,11 @@ int ha_partition::write_row(uchar * buf) > error= HA_ERR_NOT_IN_LOCK_PARTITIONS; > goto exit; > } > + > + if (table->part_info->n_uniques_to_check && > + check_uniques_insert(buf, part_id, &error)) > + goto exit; This would _probably_ introduce the race condition I've mentioned in a comment to MDEV-14005. Say, you have a table with two partitions and a unique key. Now, two threads try to insert rows, the first thread inserts 1 into the first partition (there are more columns in a table and a partitioning key on those columns, but I omit these details for simplicity). And a second thread inserts 1 into the second partition. So, the first thread checks for 1 in the second partition, finds nothing. Second thread checks for 1 in the first partition, finds nothing. Then the first thread inserts 1 into the first partition, second thread inserts 1 into the second partition. Monty's idea was to insert optimistically - first you insert, then check for duplicates. Not ideal, in the above example you can have both threads getting a duplicate key error. There could be other, e.g. locking, solutions too. > m_last_part= part_id; > DBUG_PRINT("info", ("Insert in partition %d", part_id)); > start_part_bulk_insert(thd, part_id); Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MDEV-13995 MAX(timestamp) returns a wrong result near DST change
by Alexander Barkov 04 Dec '17

04 Dec '17

Hi Sergei, Please review a patch for MDEV-13995. Thanks!

1 0

[Maria-developers] Review for MDEV-8320 Allow index usage for DATE(datetime_column) = const
by Alexander Barkov 30 Nov '17

30 Nov '17

Hello Sergey, I generally like your patch for MDEV-8320: http://lists.askmonty.org/pipermail/commits/2017-November/011680.html However, there is one problem with it. We're changing the code to have data type plugins. Every data type plugin will provide its own list of SQL functions. Those data type specific functions should be able to use the same optimization approach without having to change the server code. Therefore, everything related to YEAR(field) transformation should reside inside Item_func_year, everything related to DATE(field) transformation should reside inside Item_date_typecast. I reorganized the code to implement this idea. Note, also I changed the logic behind DATE(indexed_date_field) transformation slightly. The DATE() just gets removed. DATE(indexed_date_field) = const is changed to: indexed_date_field = const instead of: indexed_date_field BETWEEN ... AND ... Please have a look: sargable_date_cond.result is slightly different in my version. Greetings.

1 1

[Maria-developers] Delete vs multi_delete performance issue
by jerome brauge 29 Nov '17

29 Nov '17

Hi All, I've a question on kind of delete on whose we have performance issues. Consider two tables: CREATE TABLE `gapfl` ( `pgegapfl` varchar(150) COLLATE utf8_bin NOT NULL, `blkgapfl` varchar(30) COLLATE utf8_bin NOT NULL, `fldgapfl` varchar(30) COLLATE utf8_bin NOT NULL, `ordgapfl` int(11) NOT NULL, ... `stlgapfl` varchar(30) COLLATE utf8_bin DEFAULT NULL, UNIQUE KEY `gapfl_fld` (`pgegapfl`,`blkgapfl`,`fldgapfl`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin and CREATE TABLE `gapge` ( `nuugapge` varchar(150) COLLATE utf8_bin NOT NULL, `nuigapge` int(11) NOT NULL, `frmgapge` varchar(30) COLLATE utf8_bin NOT NULL, `prsgapge` varchar(60) COLLATE utf8_bin NOT NULL, `typgapge` varchar(2) COLLATE utf8_bin NOT NULL, `catgapge` varchar(1) COLLATE utf8_bin NOT NULL, `numgapge` int(11) NOT NULL, `aplgapge` varchar(8) COLLATE utf8_bin DEFAULT NULL, `tragapge` varchar(8) COLLATE utf8_bin DEFAULT NULL, ... `mafgapge` varchar(1) COLLATE utf8_bin NOT NULL, UNIQUE KEY `gapge_nuu` (`nuugapge`), UNIQUE KEY `gapge_frm` (`frmgapge`,`prsgapge`,`typgapge`,`numgapge`,`tragapge`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin Table "gapfl " contains 364740 rows Table "gapge " contains 25423 rows The following delete tooks more than 38 seconds to delete 0 rows . DELETE FROM gapfl WHERE pgegapfl IN ( SELECT nuugapge FROM gapge WHERE frmgapge = 'AMCEVA' AND prsgapge = 'QUALIAC.*.*') It explain plan is : +------+--------------------+-------+-----------------+---------------------+-----------+---------+------+--------+-------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +------+--------------------+-------+-----------------+---------------------+-----------+---------+------+--------+-------------+ | 1 | PRIMARY | gapfl | ALL | NULL | NULL | NULL | NULL | 356417 | Using where | | 2 | DEPENDENT SUBQUERY | gapge | unique_subquery | gapge_nuu,gapge_frm | gapge_nuu | 452 | func | 1 | Using where | +------+--------------------+-------+-----------------+---------------------+-----------+---------+------+--------+-------------+ It's not really good and I don't understand why the subselect is consider as a "DEPENDENT SUBQUERY" (there is no correlation columns). Now, I convert this delete into a multi_delete : DELETE gapfl FROM gapfl WHERE pgegapfl IN ( SELECT nuugapge FROM gapge WHERE frmgapge = 'AMCEVA' AND prsgapge = 'QUALIAC.*.*') It tooks 0.00 second and it explain plan is : +------+-------------+-------+------+---------------------+-----------+---------+--------------------+------+--------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +------+-------------+-------+------+---------------------+-----------+---------+--------------------+------+--------------------------+ | 1 | PRIMARY | gapge | ref | gapge_nuu,gapge_frm | gapge_frm | 274 | const,const | 1 | Using where; Using index | | 1 | PRIMARY | gapfl | ref | gapfl_fld | gapfl_fld | 452 | dvt.gapge.nuugapge | 7 | | +------+-------------+-------+------+---------------------+-----------+---------+--------------------+------+--------------------------+ as expected. Is there any way to have the right plan with single delete ? Or do we must convert all our complex update to multi_update ? Sorry if it's not the right place to post this query. Regards.

1 0

[Maria-developers] Error 1422 - Explicit or implicit commit is not allowed in stored function or trigger
by jerome brauge 29 Nov '17

29 Nov '17

Hello Alexander, I'm facing to a new issue with my stored procedures. Some of them are called directly or from trigger. When these procedures are called directly, they manage the transaction and when they are called from trigger, they doesn't manage the transaction. This works fine on Oracle, SQLServer , ... but fail on Mariadb because the check is done in is_not_allowed_in_function() before the call. use test; set sql_mode=oracle; create or replace table t1 (c1 int); insert into t1 values (1); delimiter / create or replace procedure p1(manage_transactionnal int) is begin if manage_transactionnal = 1 then commit; end if; end; / create or replace trigger tg1 after delete on t1 for each row begin call p1(0); end; / delimiter ; start transaction; delete from t1; rollback; What do you think about this ? Regards, Jérôme.

2 2

[Maria-developers] MariaDB 10.2 now available on CentOS 6 &7 via SCL !!
by Michal Schorm 29 Nov '17

29 Nov '17

Hello everybody, In case you missed it, I just want to share an announce of availability of MariaDB 10.2 on CentOS 6 & 7 as a Software collections. Here are the announce emails from CentOS mailing list, which contains most basic howto: https://lists.centos.org/pipermail/centos-announce/2017-November/022640.html https://lists.centos.org/pipermail/centos-announce/2017-November/022642.html -- This made MariaDB 10.2 a big step closer to a new mass of users, because the standard MDB version in CentOS & RHEL is 5.5. It took quite an effort to fine-tune and perk-up the MDB 10.2 since its first attemps in Fedora, however it is always pleasure to cooperate with developers like you :) -- Michal Schorm Associate Software Engineer Core Services - Databases Team Red Hat

1 0

[Maria-developers] Interaction between rpl_slave_state and rpl_binlog_state
by Sachin Setiya 28 Nov '17

28 Nov '17

Hi Kristian, Andrei I have some question related to rpl_slave_state. Suppose A circular async replication between A < -- > B (gtid_ignore_duplicates on) Now, we set some temp server_id on server A , lets say `X`. Now the problem is each event group which originates from A is executed 2 times. For example we insert into table t1 and gtid is 0-X-2. The event goes to slave B B applies it, And send it back to A, Since its server_is different from A global server_id , this event is not filtered in queue_event(). Now event goes to handle_sql thread , And check_duplicate_gtid function is called. Since rpl_slave_state::hash does not have a element whose sequence_no is >= 2, So this event is applied again. Andrei suggested a solution of checking rpl_binlog_state in check_duplicate_gtid, This solution solves some problem but creates some problem. (multi_source.gtid_ignore_duplicates fails in master_sync, test for mdev_10715 also fails ). Reason for this is sync_with_master calls master_gtid_wait , which internally calls gtid_waiting::wait_for_gtid , and this checks rpl_salve_state::hash. Although this method can also be patched for to look for rpl_gtid_global_binlog_state. But I am not sure whether this will solve all problem or create new problem. Another solution might be somehow update rpl_slave_state::hash when we write gtid_event in log. But this does not make sense. rpl_slave_state should be used for slave replication usage. I think we need a more better solution for this. -- Regards sachin

3 4

[Maria-developers] Add hidden level 2/3 (PSEDUO_HIDDEN_COLUMN / COMPLETELY_HIDDEN_COLUMN) to mariadb
by Sachin Setiya 27 Nov '17

27 Nov '17

Hello Everyone! Currently I am implementing 10177. But I have one doubt I am not sure how to implement a function which will create hidden level 2/3 field for server. Suppose we create a function (say add_hidden_field(.....)) which will add the field to table T. But the issue is where should we call this function in mysql_prepare_create_table / mysql_prepare_alter_table or init_from_binary_frm_image Doing it from init_from_binary_frm_image make sense because this way server and slave frm of table T will be same (We have decided that we wont log hidden level 2/3 field to binlog) , But the issue is optimizer wont be ready to decide whether we require a hidden level 3 field at that time. We can also do it from mysql_prepare_create_table , but I dont think this wont be good idea because at this time optimizer wont be ready plus this will complicate binlog handling. Or may be we should have a function which can modify table object any time. But I am not sure if it is possible/good idea ? What you guys think ? -- Regards Sachin Setiya Software Engineer at MariaDB

2 4

Re: [Maria-developers] [Commits] b2571e5: Removed ut_crc32_byte_by_byte: never used
by Marko Mäkelä 23 Nov '17

23 Nov '17

Indeed, the functions and function pointers were never used. On Thu, Nov 23, 2017 at 12:36 PM, Sergey Vojtovich <svoj(a)mariadb.org> wrote: > revision-id: b2571e534f9eed873cc9b1ce5f48b109c1ee0fcc > (mariadb-10.3.2-65-gb2571e5) > parent(s): 0bfe5e349210c01404d20dfc8660d5224bd9c658 > committer: Sergey Vojtovich > timestamp: 2017-11-23 14:35:50 +0400 > message: > > Removed ut_crc32_byte_by_byte: never used > > --- > storage/innobase/include/ut0crc32.h | 4 ---- > storage/innobase/ut/ut0crc32.cc | 48 ------------------------------ > ------- > 2 files changed, 52 deletions(-) > > diff --git a/storage/innobase/include/ut0crc32.h > b/storage/innobase/include/ut0crc32.h > index 36b389b..509ac09 100644 > --- a/storage/innobase/include/ut0crc32.h > +++ b/storage/innobase/include/ut0crc32.h > @@ -51,10 +51,6 @@ extern ut_crc32_func_t ut_crc32; > when converting byte strings to integers internally. */ > extern ut_crc32_func_t ut_crc32_legacy_big_endian; > > -/** Pointer to CRC32-byte-by-byte calculation function (byte order > agnostic, > -but very slow). */ > -extern ut_crc32_func_t ut_crc32_byte_by_byte; > - > extern const char* ut_crc32_implementation; > > #endif /* ut0crc32_h */ > diff --git a/storage/innobase/ut/ut0crc32.cc b/storage/innobase/ut/ > ut0crc32.cc > index 386d32f..6383e6d 100644 > --- a/storage/innobase/ut/ut0crc32.cc > +++ b/storage/innobase/ut/ut0crc32.cc > @@ -93,10 +93,6 @@ ut_crc32_func_t ut_crc32; > when converting byte strings to integers internally. */ > ut_crc32_func_t ut_crc32_legacy_big_endian; > > -/** Pointer to CRC32-byte-by-byte calculation function (byte order > agnostic, > -but very slow). */ > -ut_crc32_func_t ut_crc32_byte_by_byte; > - > /** Text description of CRC32 implementation */ > const char* ut_crc32_implementation; > > @@ -409,26 +405,6 @@ ut_crc32_legacy_big_endian_hw( > > return(~crc); > } > - > -/** Calculates CRC32 using hardware/CPU instructions. > -This function processes one byte at a time (very slow) and thus it does > -not depend on the byte order of the machine. > -@param[in] buf data over which to calculate CRC32 > -@param[in] len data length > -@return CRC-32C (polynomial 0x11EDC6F41) */ > -uint32_t > -ut_crc32_byte_by_byte_hw( > - const byte* buf, > - ulint len) > -{ > - uint32_t crc = 0xFFFFFFFFU; > - > - while (len > 0) { > - ut_crc32_8_hw(&crc, &buf, &len); > - } > - > - return(~crc); > -} > #endif /* defined(__GNUC__) && defined(__x86_64__) */ > > /* CRC32 software implementation. */ > @@ -669,28 +645,6 @@ ut_crc32_legacy_big_endian_sw( > return(~crc); > } > > -/** Calculates CRC32 in software, without using CPU instructions. > -This function processes one byte at a time (very slow) and thus it does > -not depend on the byte order of the machine. > -@param[in] buf data over which to calculate CRC32 > -@param[in] len data length > -@return CRC-32C (polynomial 0x11EDC6F41) */ > -uint32_t > -ut_crc32_byte_by_byte_sw( > - const byte* buf, > - ulint len) > -{ > - uint32_t crc = 0xFFFFFFFFU; > - > - ut_a(ut_crc32_slice8_table_initialized); > - > - while (len > 0) { > - ut_crc32_8_sw(&crc, &buf, &len); > - } > - > - return(~crc); > -} > - > /********************************************************************//** > Initializes the data structures used by ut_crc32*(). Does not do any > allocations, would not hurt if called twice, but would be pointless. */ > @@ -701,7 +655,6 @@ ut_crc32_init() > ut_crc32_slice8_table_init(); > ut_crc32 = ut_crc32_sw; > ut_crc32_legacy_big_endian = ut_crc32_legacy_big_endian_sw; > - ut_crc32_byte_by_byte = ut_crc32_byte_by_byte_sw; > ut_crc32_implementation = "Using generic crc32 instructions"; > > #if defined(__GNUC__) && defined(__x86_64__) > @@ -735,7 +688,6 @@ ut_crc32_init() > if (features_ecx & 1 << 20) { > ut_crc32 = ut_crc32_hw; > ut_crc32_legacy_big_endian = ut_crc32_legacy_big_endian_hw; > - ut_crc32_byte_by_byte = ut_crc32_byte_by_byte_hw; > ut_crc32_implementation = "Using SSE2 crc32 instructions"; > } > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- Marko Mäkelä, Lead Developer InnoDB MariaDB Corporation DON’T MISS M|18 MariaDB User Conference February 26 - 27, 2018 New York City https://m18.mariadb.com/

1 0

Re: [Maria-developers] [Commits] f28af02: Removed HW acceleration for big endian checksum
by Marko Mäkelä 23 Nov '17

23 Nov '17

Looks OK to me. This can only slightly punish the very few people who are upgrading from a buggy MySQL 5.6 or MariaDB 10.0 or 10.1 installation on a big-endian machine (such as SPARC) into a newer version. And those users could invoke innochecksum to rewrite the page checksums. On Thu, Nov 23, 2017 at 12:57 PM, Sergey Vojtovich <svoj(a)mariadb.org> wrote: > revision-id: f28af027a0439d760c6d26eb67a932e97eb665d7 > (mariadb-10.3.2-66-gf28af02) > parent(s): b2571e534f9eed873cc9b1ce5f48b109c1ee0fcc > committer: Sergey Vojtovich > timestamp: 2017-11-23 14:53:16 +0400 > message: > > Removed HW acceleration for big endian checksum > > Big endian checksum is needed to workaround 10+ years old bug, where > checksum > was calculated incorrectly on big endian hardware. We can still checksum > such > tablespaces using software implementation of CRC32. > > --- > storage/innobase/include/ut0crc32.h | 4 +- > storage/innobase/ut/ut0crc32.cc | 89 +----------------------------- > ------- > 2 files changed, 3 insertions(+), 90 deletions(-) > > diff --git a/storage/innobase/include/ut0crc32.h > b/storage/innobase/include/ut0crc32.h > index 509ac09..527a3e81 100644 > --- a/storage/innobase/include/ut0crc32.h > +++ b/storage/innobase/include/ut0crc32.h > @@ -47,9 +47,9 @@ typedef uint32_t (*ut_crc32_func_t)(const byte* > ptr, ulint len); > /** Pointer to CRC32 calculation function. */ > extern ut_crc32_func_t ut_crc32; > > -/** Pointer to CRC32 calculation function, which uses big-endian byte > order > +/** CRC32 calculation function, which uses big-endian byte order > when converting byte strings to integers internally. */ > -extern ut_crc32_func_t ut_crc32_legacy_big_endian; > +extern uint32_t ut_crc32_legacy_big_endian(const byte* buf, ulint len); > > extern const char* ut_crc32_implementation; > > diff --git a/storage/innobase/ut/ut0crc32.cc b/storage/innobase/ut/ > ut0crc32.cc > index 6383e6d..36c52b7 100644 > --- a/storage/innobase/ut/ut0crc32.cc > +++ b/storage/innobase/ut/ut0crc32.cc > @@ -89,10 +89,6 @@ mysys/my_perf.c, contributed by Facebook under the > following license. > /** Pointer to CRC32 calculation function. */ > ut_crc32_func_t ut_crc32; > > -/** Pointer to CRC32 calculation function, which uses big-endian byte > order > -when converting byte strings to integers internally. */ > -ut_crc32_func_t ut_crc32_legacy_big_endian; > - > /** Text description of CRC32 implementation */ > const char* ut_crc32_implementation; > > @@ -238,37 +234,6 @@ ut_crc32_64_hw( > *len -= 8; > } > > -/** Calculate CRC32 over 64-bit byte string using a hardware/CPU > instruction. > -The byte string is converted to a 64-bit integer using big endian byte > order. > -@param[in,out] crc crc32 checksum so far when this function is called, > -when the function ends it will contain the new checksum > -@param[in,out] data data to be checksummed, the pointer will be > advanced > -with 8 bytes > -@param[in,out] len remaining bytes, it will be decremented with 8 */ > -inline > -void > -ut_crc32_64_legacy_big_endian_hw( > - uint32_t* crc, > - const byte** data, > - ulint* len) > -{ > - uint64_t data_int = *reinterpret_cast<const > uint64_t*>(*data); > - > -#ifndef WORDS_BIGENDIAN > - data_int = ut_crc32_swap_byteorder(data_int); > -#else > - /* Currently we only support x86_64 (little endian) CPUs. In case > - some big endian CPU supports a CRC32 instruction, then maybe we > will > - NOT need a byte order swap here. */ > -#error Dont know how to handle big endian CPUs > -#endif /* WORDS_BIGENDIAN */ > - > - *crc = ut_crc32_64_low_hw(*crc, data_int); > - > - *data += 8; > - *len -= 8; > -} > - > /** Calculates CRC32 using hardware/CPU instructions. > @param[in] buf data over which to calculate CRC32 > @param[in] len data length > @@ -355,56 +320,6 @@ ut_crc32_hw( > > return(~crc); > } > - > -/** Calculates CRC32 using hardware/CPU instructions. > -This function uses big endian byte ordering when converting byte sequence > to > -integers. > -@param[in] buf data over which to calculate CRC32 > -@param[in] len data length > -@return CRC-32C (polynomial 0x11EDC6F41) */ > -uint32_t > -ut_crc32_legacy_big_endian_hw( > - const byte* buf, > - ulint len) > -{ > - uint32_t crc = 0xFFFFFFFFU; > - > - /* Calculate byte-by-byte up to an 8-byte aligned address. After > - this consume the input 8-bytes at a time. */ > - while (len > 0 && (reinterpret_cast<uintptr_t>(buf) & 7) != 0) { > - ut_crc32_8_hw(&crc, &buf, &len); > - } > - > - while (len >= 128) { > - /* This call is repeated 16 times. 16 * 8 = 128. */ > - ut_crc32_64_legacy_big_endian_hw(&crc, &buf, &len); > - ut_crc32_64_legacy_big_endian_hw(&crc, &buf, &len); > - ut_crc32_64_legacy_big_endian_hw(&crc, &buf, &len); > - ut_crc32_64_legacy_big_endian_hw(&crc, &buf, &len); > - ut_crc32_64_legacy_big_endian_hw(&crc, &buf, &len); > - ut_crc32_64_legacy_big_endian_hw(&crc, &buf, &len); > - ut_crc32_64_legacy_big_endian_hw(&crc, &buf, &len); > - ut_crc32_64_legacy_big_endian_hw(&crc, &buf, &len); > - ut_crc32_64_legacy_big_endian_hw(&crc, &buf, &len); > - ut_crc32_64_legacy_big_endian_hw(&crc, &buf, &len); > - ut_crc32_64_legacy_big_endian_hw(&crc, &buf, &len); > - ut_crc32_64_legacy_big_endian_hw(&crc, &buf, &len); > - ut_crc32_64_legacy_big_endian_hw(&crc, &buf, &len); > - ut_crc32_64_legacy_big_endian_hw(&crc, &buf, &len); > - ut_crc32_64_legacy_big_endian_hw(&crc, &buf, &len); > - ut_crc32_64_legacy_big_endian_hw(&crc, &buf, &len); > - } > - > - while (len >= 8) { > - ut_crc32_64_legacy_big_endian_hw(&crc, &buf, &len); > - } > - > - while (len > 0) { > - ut_crc32_8_hw(&crc, &buf, &len); > - } > - > - return(~crc); > -} > #endif /* defined(__GNUC__) && defined(__x86_64__) */ > > /* CRC32 software implementation. */ > @@ -600,7 +515,7 @@ integers. > @param[in] len data length > @return CRC-32C (polynomial 0x11EDC6F41) */ > uint32_t > -ut_crc32_legacy_big_endian_sw( > +ut_crc32_legacy_big_endian( > const byte* buf, > ulint len) > { > @@ -654,7 +569,6 @@ ut_crc32_init() > { > ut_crc32_slice8_table_init(); > ut_crc32 = ut_crc32_sw; > - ut_crc32_legacy_big_endian = ut_crc32_legacy_big_endian_sw; > ut_crc32_implementation = "Using generic crc32 instructions"; > > #if defined(__GNUC__) && defined(__x86_64__) > @@ -687,7 +601,6 @@ ut_crc32_init() > > if (features_ecx & 1 << 20) { > ut_crc32 = ut_crc32_hw; > - ut_crc32_legacy_big_endian = ut_crc32_legacy_big_endian_hw; > ut_crc32_implementation = "Using SSE2 crc32 instructions"; > } > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- Marko Mäkelä, Lead Developer InnoDB MariaDB Corporation DON’T MISS M|18 MariaDB User Conference February 26 - 27, 2018 New York City https://m18.mariadb.com/

1 0

Re: [Maria-developers] [Commits] 14bd406: Less code to depend on ut_crc32_init()
by Marko Mäkelä 23 Nov '17

23 Nov '17

Thanks, this looks OK to me. I guess the function ut_crc32_init() will be made optional separately. On Thu, Nov 23, 2017 at 1:22 PM, Sergey Vojtovich <svoj(a)mariadb.org> wrote: > revision-id: 14bd406970c9bb18c744a452359aa4505f325b2d > (mariadb-10.3.2-67-g14bd406) > parent(s): f28af027a0439d760c6d26eb67a932e97eb665d7 > committer: Sergey Vojtovich > timestamp: 2017-11-23 15:22:10 +0400 > message: > > Less code to depend on ut_crc32_init() > > --- > storage/innobase/include/ut0crc32.h | 1 + > storage/innobase/ut/ut0crc32.cc | 20 +++++++------------- > 2 files changed, 8 insertions(+), 13 deletions(-) > > diff --git a/storage/innobase/include/ut0crc32.h > b/storage/innobase/include/ut0crc32.h > index 527a3e81..32ad066 100644 > --- a/storage/innobase/include/ut0crc32.h > +++ b/storage/innobase/include/ut0crc32.h > @@ -51,6 +51,7 @@ extern ut_crc32_func_t ut_crc32; > when converting byte strings to integers internally. */ > extern uint32_t ut_crc32_legacy_big_endian(const byte* buf, ulint len); > > +/** Text description of CRC32 implementation */ > extern const char* ut_crc32_implementation; > > #endif /* ut0crc32_h */ > diff --git a/storage/innobase/ut/ut0crc32.cc b/storage/innobase/ut/ > ut0crc32.cc > index 36c52b7..48f6b82 100644 > --- a/storage/innobase/ut/ut0crc32.cc > +++ b/storage/innobase/ut/ut0crc32.cc > @@ -86,12 +86,6 @@ mysys/my_perf.c, contributed by Facebook under the > following license. > #include "univ.i" > #include "ut0crc32.h" > > -/** Pointer to CRC32 calculation function. */ > -ut_crc32_func_t ut_crc32; > - > -/** Text description of CRC32 implementation */ > -const char* ut_crc32_implementation; > - > /** Swap the byte order of an 8 byte integer. > @param[in] i 8-byte integer > @return 8-byte integer */ > @@ -125,6 +119,13 @@ ut_crc32_power8( > { > return crc32c_vpmsum(0, buf, len); > } > + > +ut_crc32_func_t ut_crc32 = ut_crc32_power8; > +const char* ut_crc32_implementation = "Using POWER8 crc32 > instructions"; > +#else > +uint32_t ut_crc32_sw(const byte* buf, ulint len); > +ut_crc32_func_t ut_crc32 = ut_crc32_sw; > +const char* ut_crc32_implementation = "Using generic crc32 > instructions"; > #endif > > #if defined(__GNUC__) && defined(__x86_64__) > @@ -568,8 +569,6 @@ ut_crc32_init() > /*===========*/ > { > ut_crc32_slice8_table_init(); > - ut_crc32 = ut_crc32_sw; > - ut_crc32_implementation = "Using generic crc32 instructions"; > > #if defined(__GNUC__) && defined(__x86_64__) > uint32_t vend[3]; > @@ -603,10 +602,5 @@ ut_crc32_init() > ut_crc32 = ut_crc32_hw; > ut_crc32_implementation = "Using SSE2 crc32 instructions"; > } > - > -#elif defined(HAVE_CRC32_VPMSUM) > - ut_crc32 = ut_crc32_power8; > - ut_crc32_implementation = "Using POWER8 crc32 instructions"; > #endif > - > } > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- Marko Mäkelä, Lead Developer InnoDB MariaDB Corporation DON’T MISS M|18 MariaDB User Conference February 26 - 27, 2018 New York City https://m18.mariadb.com/

1 0

[Maria-developers] IB: trx_t:: start_time and start_time_micro
by Aleksey Midenkov 21 Nov '17

21 Nov '17

Hello! Different semantics of these 2 members doesn't look good: time_t start_time; /*!< time the state last time became TRX_STATE_ACTIVE */ ib_uint64_t start_time_micro; /*!< start time of transaction in microseconds */ Either `start_time_micro` should be drawn to the base of `start_time` or it should be explicitly commented that it has nothing to do with `start_time`. I.e. there is nothing wrong on algorithmic level, but such data members may (and certainly would) lead developers to misuse of `start_time_micro`. -- All the best, Aleksey Midenkov @midenok

2 2

[Maria-developers] Please review MDEV-11292 Item_func_sp::fix_length_and_dec() does not set collation derivation properly
by Alexander Barkov 20 Nov '17

20 Nov '17

Hello Alexey, Please review a patch for MDEV-11292 (for 10.3). Thanks.

2 2

[Maria-developers] MariaDB GUI: BSL 1.1 - Debate
by Juan Telleria 19 Nov '17

19 Nov '17

Dear MariaDb Developers, I am writing because I would like to open a debate whether if MariaDB, now that has gained investment from Microsoft, Alibaba, and the European Union, shall create its own SQL Graphical User Interface, as good as (or better than) MySQL Workbench. I agree that MariaDB includes by default HeidiSQL in its Windows builds, but in my opinion, if MariaDB really wants to look Oracle's MySQL face to face, HeidiSQL is not nearly as good as MySQL Workbench. If this wants to be done: 1st) The brand new GUI shall have a Business Source Licence v1.1, which only companies which invest in MariaDB would be able to use. 2nd) In 3 years now, MariaDB would be better positioned and have a fully Open Source GUI which would allow MariaDB to look face to face to Oracle. 3rd) And in 4/5 years now, MariaDB Corporation shall be brave enough to to continue Developing this GUI as a truly open source product, without the BSL, and adopting a purely GPLv2 with incremental improvements, distributing it through the MariaDB Foundation Webpage, as individuals or other companies apart from MariaDB Corporation, will not support the product if it is not totally Open Source. So BSL v1.1 might prove useful to launch the project, but with time, it shall have a GPLv2 for gaining investment from third companies. ¿What do you thing on the topic? Thank you all, Juan Telleria

1 0

[Maria-developers] Please review MDEV-13788 Server crash when issuing bad SQL partition syntax
by Alexander Barkov 16 Nov '17

16 Nov '17

Hello Sergei, Can you please review a patch for MDEV-13788, for 5.5? Thanks!

2 3

Re: [Maria-developers] MDEV-10173.
by Sergei Golubchik 15 Nov '17

15 Nov '17

Hi, Alexey! On Nov 14, Alexey Botchkov wrote: > Hi, Serg. > > I'd like your opinion about this issue. > Georg complains about weird column metadata > https://jira.mariadb.org/browse/MDEV-10173 > > Personally i see slightly different result, but basically same thing. > -------------------------------- > MariaDB [(none)]> select from_unixtime("0"); > Field 1: `from_unixtime("0")` > Catalog: `def` > Database: `` > Table: `` > Org_table: `` > Type: DATETIME > Collation: binary (63) > Length: 26 > Max_length: 19 > Decimals: 39 > Flags: BINARY Right, I get used to 31 being NOT_FIXED_DEC. But recently Monty changed it to 39. > +---------------------+ > | from_unixtime("0") | > +---------------------+ > | 1970-01-01 04:00:00 | > +---------------------+ > 1 row in set (0.00 sec) > -------------------------------- > > From what i see it is not a bug. The 'Decimals: 39' is not a real > precision here, but just a hint to be flexible converting the data to > the string. The 'Length: 26' takes into account the possible > 'second_part' of 6 digits. Agree. > Georg said on this: "Well, the client "trusts" the metadata. If there > is decimal 31, it appends 31 decimal places and returned truncation > error the metadata (MYSQL_FIELD) describes the data which are sent > from server to client. If there are no decimal places, it should be > zero. " Yes, the cient trusts the metadata, but if decimals is NOT_FIXED_DEC it means "unknown", not 31 (or 39). If the number of decimals for a timestamp is unknown, the max length is 26: "YYYY-MM-DD hh:mm:ss.uuuuuu" > As an improvement i can see using '7' instead of '39' as an > 'NOT_FIXED_DEC' marker for the Item_temporal_func. Why? One wouldn't be able simply to compare with NOT_FIXED_DEC, it'll be type dependent. > Or we can just document that '39' value so clients interpret it right. Isn't it documented? > Serg, maybe you can suggest something? Close as "Not a bug" :) Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] 730a22c: Fixed bug mdev-13453 Executing a query via CTE requires more permissions
by Oleksandr Byelkin 14 Nov '17

14 Nov '17

Am 14.11.2017 um 01:06 schrieb IgorBabaev: > revision-id: 730a22c1ce1010e853e7a3e807d1d07409ec4dfc (mariadb-10.2.10-28-g730a22c) > parent(s): 1e2d4f677e08294691a7d218acb3b9f78727ae18 > author: Igor Babaev > committer: Igor Babaev > timestamp: 2017-11-13 16:06:04 -0800 > message: > > Fixed bug mdev-13453 Executing a query via CTE requires more permissions > than the query itself > > ACL checks were not properly supported for tables used in CTE > specifications. This patch fixes the problem. > > --- > mysql-test/r/cte_nonrecursive.result | 58 ++++++++++++++++++++++++++++++++++++ > mysql-test/t/cte_nonrecursive.test | 51 +++++++++++++++++++++++++++++++ > sql/sql_acl.cc | 4 +++ > sql/sql_cte.cc | 3 +- > sql/sql_parse.cc | 4 +++ > 5 files changed, 119 insertions(+), 1 deletion(-) > > diff --git a/mysql-test/r/cte_nonrecursive.result b/mysql-test/r/cte_nonrecursive.result > index ebe1aae..92f9151 100644 > --- a/mysql-test/r/cte_nonrecursive.result > +++ b/mysql-test/r/cte_nonrecursive.result > @@ -1147,3 +1147,61 @@ SELECT * FROM cte_test; > a > 1 > DROP VIEW cte_test; > +# > +# MDEV-13453: > +# [skip] > --- a/mysql-test/t/cte_nonrecursive.test > +++ b/mysql-test/t/cte_nonrecursive.test > @@ -790,3 +790,54 @@ SHOW CREATE VIEW cte_test; > SELECT * FROM cte_test; > > DROP VIEW cte_test; > + > +--echo # > +--echo # MDEV-13453: privileges checking for CTE > +--echo # > + This is for sure do not match results above, so please fix it and make sure that full test suite passed. [skip] > --- a/sql/sql_acl.cc > +++ b/sql/sql_acl.cc > @@ -7557,6 +7557,10 @@ bool check_grant(THD *thd, ulong want_access, TABLE_LIST *tables, > tl->correspondent_table ? tl->correspondent_table : tl; > sctx= t_ref->security_ctx ? t_ref->security_ctx : thd->security_ctx; > > + if (tl->with || > + (tl->with= tl->select_lex->find_table_def_in_with_clauses(tl))) > + continue; > + > const ACL_internal_table_access *access= > get_cached_table_access(&t_ref->grant.m_internal, > t_ref->get_db_name(), > diff --git a/sql/sql_cte.cc b/sql/sql_cte.cc > index 6fe08e3..e1bd455 100644 > --- a/sql/sql_cte.cc > +++ b/sql/sql_cte.cc > @@ -823,9 +823,10 @@ st_select_lex_unit *With_element::clone_parsed_spec(THD *thd, > tbl; > tbl= tbl->next_global) > { > - tbl->grant.privilege= with_table->grant.privilege; > spec_tables_tail= tbl; > } > + if (check_table_access(thd, SELECT_ACL, spec_tables, FALSE, UINT_MAX, FALSE)) > + goto err; > if (spec_tables) > { > if (with_table->next_global) > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index bf5144b..39a4da9 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -3443,6 +3443,10 @@ mysql_execute_command(THD *thd) > ulong privileges_requested= lex->exchange ? SELECT_ACL | FILE_ACL : > SELECT_ACL; > > + res= check_dependencies_in_with_clauses(thd->lex->with_clauses_list); > + if (res) > + break; Above works only for SELECT, do we support CTE in subselects which can be put everywhere? > + > if (all_tables) > res= check_table_access(thd, > privileges_requested, > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

Re: [Maria-developers] Handle failures from malloc
by Sergey Petrunia 14 Nov '17

14 Nov '17

Hi Monty, Please find my comments below. Anything not commented on seems to be ok to me. > commit 7d9c24f6790c66d2f457837661852bf5e870b605 > Author: Michael Widenius <monty(a)mariadb.org> > Date: Tue Nov 14 07:47:58 2017 +0200 > > Handle failures from malloc > > Most "new" failures fixed in the following files: > - sql_select.cc > - item.cc > - opt_subselect.cc > > diff --git a/sql/item.cc b/sql/item.cc > index 2285ae28041..baae2a9b6fb 100644 > --- a/sql/item.cc > +++ b/sql/item.cc > @@ -1261,7 +1261,7 @@ Item *Item::safe_charset_converter(THD *thd, CHARSET_INFO *tocs) > if (!needs_charset_converter(tocs)) > return this; > Item_func_conv_charset *conv= new (thd->mem_root) Item_func_conv_charset(thd, this, tocs, 1); > - return conv->safe ? conv : NULL; > + return conv && conv->safe ? conv : NULL; > } > I am not fully convinced that returning NULL from this function is called everywhere. It is of course better to return NULL and hope for the best than to crash right away. > @@ -3296,6 +3301,8 @@ void Item_field::fix_after_pullout(st_select_lex *new_parent, Item **ref, > } > > Name_resolution_context *ctx= new Name_resolution_context(); > + if (ctx) > + return; // Fatal error set > if (context->select_lex == new_parent) > { > /* I assume this is a typo it should be "if (!ctx)" > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index abbf2616537..7e42890143b 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -713,7 +713,7 @@ JOIN::prepare(TABLE_LIST *tables_init, > union_part= unit_arg->is_unit_op(); > > if (select_lex->handle_derived(thd->lex, DT_PREPARE)) > - DBUG_RETURN(1); > + DBUG_RETURN(-1); > > thd->lex->current_select->context_analysis_place= NO_MATTER; > thd->lex->current_select->is_item_list_lookup= 1; I assume this is just for the sake of consistency with other DBUG_RETURN statements in this function? > @@ -14422,7 +14446,7 @@ static COND* substitute_for_best_equal_field(THD *thd, JOIN_TAB *context_tab, > This works OK with PS/SP re-execution as changes are made to > the arguments of AND/OR items only > */ > - if (new_item != item) > + if (new_item && new_item != item) > li.replace(new_item); > } new_item was returned from a substitute_for_best_equal_field() call. That one doesn't ever return NULL. It will return without substitution if it encounters an error. So this change is not needed. > @@ -25044,8 +25103,11 @@ int JOIN::save_explain_data_intern(Explain_query *output, > > if (message) > { > - explain= new (output->mem_root) Explain_select(output->mem_root, > - thd->lex->analyze_stmt); > + if (!(explain= new (output->mem_root) > + Explain_select(output->mem_root, > + thd->lex->analyze_stmt))) > + DBUG_RETURN(1); > + > join->select_lex->set_explain_type(true); > > explain->select_id= join->select_lex->select_number; The return value of this function is not checked. I assume the above statement is there so that we dont try to continue to execute the function if we've got an OOM error? In this case, probably JOIN::save_explain_data_intern() should also have a similar check for its tab->save_explain_data(eta, used_tables, distinct_arg, first_top_tab); call? BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] STRICT mode and Cast
by jerome brauge 14 Nov '17

14 Nov '17

Hello Alexander, To have a behavior the nearest possible to Oracle, I use STRICT_TRANS_TABLES sql_mode. But then I have a problem with cast to varchar that failed with "truncated incorrect value" . Ex: set sql_mode='ORACLE,STRICT_TRANS_TABLES'; delimiter / CREATE or replace procedure p3(b1 integer) AS strres VARCHAR(4); BEGIN strres:=cast('123456' as char(4)); END / call p3(-1) / -- Failed with : ERROR 1292 (22007): Truncated incorrect CHAR(4) value: '123456' But select cast('11111' as varchar(2)) from dual; just issue a warning, not an error. On Oracle, Sybase,SQLServer and DB2, cast a string to a shorter string never issue warnings nor error. What do you think about ? Regards, Jérôme.

2 4

[Maria-developers] Patch for MDEV-10574
by jerome brauge 13 Nov '17

13 Nov '17

Hello Alexander, I've begun to work on new sql_mode EMPTY_STRING_IS_NULL_FUNCTIONS. Can you take a glance on this patch to known if I'm on the right direction ? I've also found a different behavior between Oracle and Mariadb on cast of string in a numeric format. Mariadb send a warning or a note and return 0 when conversion failed while Oracle send an error. MariaDB [test]> select cast(' ' as integer) from dual; +----------------------+ | cast(' ' as integer) | +----------------------+ | 0 | +----------------------+ DVTORA> select cast(' ' as integer) from dual; select cast(' ' as integer) from dual * ERROR at line 1: ORA-01722: invalid number Are "cast" and aggregate functions to be handled by this mdev? Regards, Jérôme. > -----Message d'origine----- > De : Alexander Barkov [mailto:bar@mariadb.org] > Envoyé : samedi 7 octobre 2017 11:44 > À : jerome brauge > Objet : Re: Patch for MDEV-10574 > > Hello Jerome, > > > On 10/06/2017 01:41 PM, jerome brauge wrote: > > Hello Alexander, > > > >> -----Message d'origine----- > >> De : Alexander Barkov [mailto:bar@mariadb.org] Envoyé : jeudi 5 > >> octobre 2017 21:26 À : jerome brauge Objet : Re: Patch for MDEV-10574 > >> > >> Hello Jerome, > >> > >> > >> On 10/05/2017 12:45 PM, jerome brauge wrote: > >>> Hello Alexander, > >>> Have you gone ahead on your reflection about nulls and empty strings ? > >> > >> Yes, I think so. > >> > >> One addition: Monty suggested to rename flags to: > >> - EMPTY_STRING_IS_NULL for literals > >> - EMPTY_STRING_IS_NULL_RESULTS for functions. > >> > >> I'm fine with EMPTY_STRING_IS_NULL for literals, but think that > >> EMPTY_STRING_IS_NULL_FUNCTIONS should be more clear than > >> EMPTY_STRING_IS_NULL_RESULTS. > >> Which name do you prefer for functions? > > > > I think like you. > > For me, EMPTY_STRING_IS_NULL_RESULTS sounds like including > RESULTSET > > of select (which is perhaps included in point 4) > > Okey, let's go with _FUNCTIONS then. > > > > >>> > >>> Can I > >>> - redo my pull request for substring_oracle function > >>> - make a patch for first new sql_mode > >> (MODE_EMPTY_AS_NULL_LITERALS) > >> > >> > >> Yes, please. Let's move forward. > >> > >> Let's push the patch for substr_oracle first. > >> Let's keep it return empty strings for the cases when the > >> substring_length parameters is less than 1. > >> It will start automatically returning NULL when we add the new > >> sql_mode for functions. > > > > Pull request is done. > > I have merged it. Thanks! > > > > >> Also, let's go ahead with the part for literals. > > > > Ok, I will make a separate patch. > > Many thanks. > > > >> > >> > >> I have created MDEVs. Please use them in the commit comment and in > >> the > >> tests: > >> > >> MDEV-14012 sql_mode=Oracle: substr(): treat position 0 as position 1 > >> MDEV-14013 sql_mode=EMPTY_STRING_IS_NULL > >> > >> > >> Thanks! > >> > >>> > >>> Regards, > >>> Jérôme. > >>> > >>> > >>>> -----Message d'origine----- > >>>> De : Alexander Barkov [mailto:bar@mariadb.org] Envoyé : mercredi 27 > >>>> septembre 2017 08:50 À : jerome brauge Objet : Re: Patch for > >>>> MDEV-10574 > >>>> > >>>> Hello Jerome, > >>>> > >>>> I told to Monty yesterday. > >>>> > >>>> We were going to implement the following transformations: > >>>> > >>>>> We are considering to implement both of the following options > >>>>> (with a > >>>>> switch) > >>>>> > >>>>> Transform Insert > >>>>> - insert values ("") -> insert values (null) > >>>>> > >>>>> Transform Select > >>>>> > >>>>> where v=x => (v <> "" and V=X) > >>>>> where v is null => (v="" or v is null) > >>>>> > >>>> > >>>> See MDEV-10574 for details. > >>>> > >>>> Monty did not fully understand the idea of translating literals and > >> functions > >>>> from empty strings to null. He asked if you can explain use cases > >>>> for these transformations. > >>>> > >>>> But we agreed that: > >>>> > >>>> 1. It's OK to have multiple NULL handling flags in sql_mode. > >>>> > >>>> 2. New flags will not be enabled in 10.3, neither in > >>>> sql_mode=DEFAULT, > >> nor in > >>>> sql_mode=ORACLE. So one will have to specify them explicitly: > >>>> set > >>>> > >> > sql_mode='ORACLE,MODE_null_transfrormation_flag1,MODE_null_transfor > >>>> mation_flag2'; > >>>> > >>>> > >>>> > >>>> So I propose we introduce the following flags: > >>>> > >>>> > >>>> 1. MODE_EMPTY_AS_NULL_LITERALS - for literals and PS parameters, > as > >> in > >>>> your patch. > >>>> > >>>> 2. MODE_EMPTY_AS_NULL_FUNCTIONS - for functions. > >>>> This should be done as a separate patch. > >>>> Note, your patch currently implements empty-to-null translation > >>>> only for Item_str_func descendants that use make_empty_result(). > >>>> This patch should be extended to cover Item_str_func descendants > >>>> which do not use make_empty_result(), as well as > >>>> all other Item_func_or_sum descendants > >>>> (which are not Item_str_func descendants). > >>>> > >>>> > >>>> 3. MODE_EMPTY_AS_NULL_UPDATES > >>>> > >>>> For INSERT and UPDATE transformation, as in MDEV: > >>>> > >>>> insert values ("") -> insert values (null) > >>>> update set a='' -> update set a=null; > >>>> > >>>> > >>>> 4. MODE_EMPTY_AS_NULL_PREDICATES > >>>> > >>>> For predicate transformation, as in MDEV: > >>>> > >>>> where v=x => (v <> "" and V=X) > >>>> where v is null => (v="" or v is null) > >>>> > >>>> > >>>> What do you think about this proposal with multiple flags? > >>>> Note, we can do N1 and N2 now, and add N3 and N4 and later. > >>>> > >>>> Also, can you please write a description why you need N1 and N2. > >>>> Monty thinks that N3 and N4 should cover most use cases. > >>>> So we need a good rationale for Monty and Sergei explaining why we > >> adding > >>>> these flags N1 and N2. > >>>> > >>>> > >>>> Thanks! > >>>> > >>>> > >>>> On 09/26/2017 04:33 PM, jerome brauge wrote: > >>>>> Hi Alexander, > >>>>> I'm disappointed. > >>>>> Can we at least keep this part in sql_prepare : > >>>>> > >>>>> /* > >>>>> set_param_str_oracle : convert empty string to null value > >>>>> This allow to bind empty string in JDBC as a null value. > >>>>> Ex : > >>>>> String sel="select coalesce(?,'null param') from dual"; > >>>>> PreparedStatement ps_sel = m_cnx.prepareStatement(sel); > >>>>> ps_sel.setString(1, ""); > >>>>> ResultSet rs = ps_sel.executeQuery(); > >>>>> while (rs.next()) > >>>>> { > >>>>> System.out.println(rs.getString(1)); > >>>>> } > >>>>> Returns : 'null param' > >>>>> */ > >>>>> static void set_param_str_oracle(Item_param *param, uchar **pos, > >> ulong > >>>>> len) { > >>>>> ulong length= get_param_length(pos, len); > >>>>> if (length == 0) > >>>>> param->set_null(); > >>>>> else > >>>>> { > >>>>> if (length > len) > >>>>> length= len; > >>>>> param->set_str((const char *) *pos, length); > >>>>> *pos+= length; > >>>>> } > >>>>> } > >>>>> > >>>>> Regards, > >>>>> Jérôme. > >>>>> > >>>>> > >>>>>> -----Message d'origine----- > >>>>>> De : Alexander Barkov [mailto:bar@mariadb.org] Envoyé : mardi 26 > >>>>>> septembre 2017 13:34 À : jerome brauge Objet : Re: Patch for > >>>>>> MDEV-10574 > >>>>>> > >>>>>> Hello Jérôme, > >>>>>> > >>>>>> I'm afraid we cannot accept this change at this point of time. > >>>>>> > >>>>>> The problem is that all around the code we assume that NULL is > >>>>>> not equal to ''. > >>>>>> > >>>>>> Changing '' to NULL only in the visible part of the iceberg (such > >>>>>> as in the parser and in Item_str_func::make_empty_result()) > >>>>>> will likely introduce various anomalies in fundamental things > >>>>>> like joins execution, optimizer, equality propagation, etc. > >>>>>> > >>>>>> Sorry, we don't have resources to dive into this topic deeper now. > >>>>>> Moreover, Oracle's documentation suggests not to reply on the > >>>>>> fact > >> that '' > >>>>>> and NULL are treated as same, as this can change in the future. > >>>>>> > >>>>>> > >>>>>> > >>>>>> On 09/20/2017 03:52 PM, jerome brauge wrote: > >>>>>>> Hello Alexander, > >>>>>>> Here is a new version of this patch (minor optimizations) > >>>>>>> > >>>>>>> Regards, > >>>>>>> Jérôme. > >>>>>>> > >>>>>>> > >>>>>>>> -----Message d'origine----- > >>>>>>>> De : Maria-developers [mailto:maria-developers- > >>>>>>>> bounces+j.brauge=qualiac.com(a)lists.launchpad.net] De la part > de > >>>>>>>> bounces+jerome > >>>>>>>> brauge > >>>>>>>> Envoyé : mardi 19 septembre 2017 16:09 À : 'Alexander Barkov' > >>>>>>>> Cc : MariaDB Developers (maria-developers(a)lists.launchpad.net) > >>>>>>>> Objet : [Maria-developers] Patch for MDEV-10574 > >>>>>>>> > >>>>>>>> Alexander, > >>>>>>>> Here is the patch I was talking about. > >>>>>>>> It not address all aspects of MDEV-10574, especially, empty > >>>>>>>> strings in table column are not view as null. > >>>>>>>> But as in oracle mode, no empty string in columns should be > >>>>>>>> created, an fully oracle application will never fall in this case. > >>>>>>>> > >>>>>>>> Regards, > >>>>>>>> Jérôme. > >>>>>>>> > >>>>>>>> > >>>>>>>>> -----Message d'origine----- > >>>>>>>>> De : jerome brauge > >>>>>>>>> Envoyé : mardi 19 septembre 2017 14:13 À : 'Alexander Barkov' > >>>>>>>>> Objet : RE: Oracle strings functions compatibility : substr > >>>>>>>>> > >>>>>>>>> Hello Alexander, > >>>>>>>>> Yes I'm aware. > >>>>>>>>> This point will be treated by my next patch which will also > >>>>>>>>> affect the others string functions (a part of MDEV-10574). > >>>>>>>>> I will refresh this new patch with the current development > >>>>>>>>> branch, and I submit to you. > >>>>>>>>> Regards. > >>>>>>>>> Jérôme. > >>>>>>>>> > >>>>>>>>>> -----Message d'origine----- > >>>>>>>>>> De : Alexander Barkov [mailto:bar@mariadb.org] Envoyé : > mardi > >> 19 > >>>>>>>>>> septembre 2017 13:16 À : jerome brauge Objet : Re: Oracle > >>>>>>>>>> strings functions compatibility : substr > >>>>>>>>>> > >>>>>>>>>> Hi Jerome, > >>>>>>>>>> > >>>>>>>>>> I noticed one more difference in SUBSTR: > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> SELECT SUBSTR('aaa',1,-1) FROM DUAL; > >>>>>>>>>> > >>>>>>>>>> MariaDB returns an empty string. > >>>>>>>>>> Oracle returns NULL. > >>>>>>>>>> > >>>>>>>>>> Here's Oracle's documentation: > >>>>>>>>>> > >>>>>>>>>> > >>>>>> > >> https://docs.oracle.com/cd/B19306_01/server.102/b14200/functions162. > >>>>>>>>>> ht > >>>>>>>>>> m > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> I think we should do this in the same patch, to avoid > >>>>>>>>>> behavior change in the future. > >>>>>>>>>> > >>>>>>>>>> Would you like to try doing this additional change? > >>>>>>>>>> Or would you like me to make an incremental patch on top of > >> yours? > >>>>>>>>>> > >>>>>>>>>> Thanks! > >>>>>>>>>> > >>>>>>>>>> On 09/19/2017 10:33 AM, jerome brauge wrote: > >>>>>>>>>>> Hello Alexander, > >>>>>>>>>>> It's done. > >>>>>>>>>>> > >>>>>>>>>>> Do you have news from Sergei for MDEV-12874, MDEV-13417 > >> and > >>>>>>>> MDEV- > >>>>>>>>>> 13418 ? > >>>>>>>>>> > >>>>>>>>>> I asked Sergei, awaiting for his answer. I'll let you know. > >>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Thank you very much. > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> -----Message d'origine----- De : Alexander Barkov > >>>>>>>>>>>> [mailto:bar@mariadb.org] Envoyé : lundi > >>>>>>>>>>>> 18 septembre 2017 18:12 À : jerome brauge Objet : Re: > >>>>>>>>>>>> Oracle strings functions compatibility : substr > >>>>>>>>>>>> > >>>>>>>>>>>> Hello Jerome, > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> On 09/18/2017 05:14 PM, jerome brauge wrote: > >>>>>>>>>>>>> Hello Alexander, > >>>>>>>>>>>>> I don't understand how I've missed such an obvious > solution ! > >>>>>>>>>>>>> Here is the new patch. > >>>>>>>>>>>> > >>>>>>>>>>>> Thanks. Now it looks simpler. > >>>>>>>>>>>> > >>>>>>>>>>>> Can you please do one small thing: > >>>>>>>>>>>> move get_position() from "public:" to "protected:". > >>>>>>>>>>>> > >>>>>>>>>>>> Ok to make a pull request after this change. > >>>>>>>>>>>> > >>>>>>>>>>>> Thank you very much! > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> Regards. > >>>>>>>>>>>>> Jérôme. > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> -----Message d'origine----- De : Alexander Barkov > >>>>>>>>>>>>>> [mailto:bar@mariadb.org] Envoyé : > >> lundi > >>>>>>>>>>>>>> 18 septembre 2017 11:30 À : jerome brauge Objet : Re: > >> Oracle > >>>>>>>>>>>>>> strings functions compatibility : substr > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Hello Jerome, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On 08/17/2017 04:43 PM, jerome brauge wrote: > >>>>>>>>>>>>>>> Hello Alexander, > >>>>>>>>>>>>>>> Here is a patch for function SUBSTR in sql_mode=oracle > >>>>>>>>>>>>>>> (If position is 0, > >>>>>>>>>>>>>> then it is treated as 1). > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> I'm thinking of how to get a smaller patch. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Wouldn't it be possible to introduce: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> virtual longlong get_position(); > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> longlong Item_func_substr::get_position() { > >>>>>>>>>>>>>> return args[1]->val_int(); } > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> longlong Item_func_substr_oracle::get_position() > >>>>>>>>>>>>>> { > >>>>>>>>>>>>>> longlong pos= args[1]->val_int(); > >>>>>>>>>>>>>> return pos == 0 ? 1 : pos; } > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> and just replace all calls for args[1]->val_int() to > >> get_position() > >>>> ? > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Thanks! > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Regards, > >>>>>>>>>>>>>>> Jérôme. > >>>>>>>>>>>>>>>

2 3

Re: [Maria-developers] [Commits] f87dddaaa95: MDEV-7865: Server crashes in Item_equal_iterator<List_iterator_fast, Item>::get_curr_field on query
by Sergey Petrunia 09 Nov '17

09 Nov '17

On Sat, Oct 28, 2017 at 12:47:03AM +0530, Varun wrote: > revision-id: f87dddaaa95f9e7fc597c8839b539cc6f816c750 (mariadb-5.5.56-91-gf87dddaaa95) > parent(s): fb5fe497e526f0677a7e2f86f34237e7efd4b806 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2017-10-28 00:46:43 +0530 > message: > > MDEV-7865: Server crashes in Item_equal_iterator<List_iterator_fast, Item>::get_curr_field on query > with impossible condition and OR/AND expressions > > When multiple-equalites are merged into one in the function Item_equal::merge_with_check, then the fields that are merged from one > equality to the other still point to the previous mulitple_equality while the should be pointing to the new equality > after the merge is done .. > diff --git a/sql/item_cmpfunc.cc b/sql/item_cmpfunc.cc > index ebe088e5092..8c79359ce88 100644 > --- a/sql/item_cmpfunc.cc > +++ b/sql/item_cmpfunc.cc > @@ -5832,7 +5832,18 @@ bool Item_equal::merge_with_check(Item_equal *item, bool save_merged) > if (intersected) > { > if (!save_merged) > + { > + Item *item_it; > + fi.rewind(); > + while ((item_it= fi++)) > + { Wrong indentation > + if (!contains(fi.get_curr_field())) > + { > + item_it->set_item_equal(this); I am not sure if this can actually make a difference, but is there any reason we don't just call set_item_equal unconditionally here? My concern is that f.get_curr_field() checks for Field* objects, while set_item_equal() call applies to Items (either Item_field or Item_direct_view_ref) What if there are two different Item objects that refer to the same field? (this looks like it should not happen but I am not 100% certain about this). > + } > + } > merge(item); > + } > else > { > Item *c= item->get_const(); > @@ -5843,9 +5854,12 @@ bool Item_equal::merge_with_check(Item_equal *item, bool save_merged) > Item *item; > fi.rewind(); > while ((item= fi++)) > - { > + { > if (!contains(fi.get_curr_field())) > + { > add(item); > + item->set_item_equal(this); > + } here, we have save_merged=true, that is, the old Item_equal will still be used. Is this ok that we call set_item_equal($new_equality) for its members? If an object is a participant in two Item_equal, which one should it point to? the "most specific" one? I'll try to ask Igor about this today. BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 1

[Maria-developers] MDEV-13418 Compatibility: The order of evaluation of SELECT..INTO assignments
by jerome brauge 09 Nov '17

09 Nov '17

Hello Alexander. I've begun to implement your proposal but now I'm not sure that it's a better solution than mine. Let me explain . - first : number of temporary variables can be significant because we don't know when they are really needed and their scope are local to the statement. declare b1 INTEGER; declare res INTEGER; ... if b1 = 0 then select 1,b1+1 into b1, res from dual; end if; if b1 = 1 then select 2,b1+2 into b1, res from dual; end if; will be transform in : declare b1 INTEGER; declare res INTEGER; ... if b1 = 0 then declare _b1 INTEGER default res; declare _res INTEGER default res; select 1,b1+1 into _b1, _res from dual; set b1=_b1; set res=_res; -- _res is not needed, but we don't know because the select statement is not parsed end if; if b1 = 1 then declare _b1 INTEGER default res; declare _res INTEGER default res; select 2,b1+2 into b1, res from dual; set b1=_b1; set res=_res; -- same thing here, and we have declare two variables for each target variables end if; Perhaps we could - declare these temporary variables only one time in the first frame of the stored procedure (may be tricky) - parse columns of each select to know what variables are really assigned and reused (heavy cost in cpu and time) - second : if we can't determine variables which must have a temporary variable, number of sp_instr_set and sp_instr_set_var will be too high and their cpu cost is not acceptable. My first solution has a fixed memory impact (and memory is not an issue nowadays), and especially a very light cpu cost. What do you think about ? Regard, Jérôme. > -----Message d'origine----- > De : Alexander Barkov [mailto:bar@mariadb.org] > Envoyé : lundi 30 octobre 2017 10:02 > À : jerome brauge > Objet : Re: MDEV-14139 Anchored data types for variables > > > > On 10/28/2017 07:29 PM, Alexander Barkov wrote: > > > > On 10/27/2017 10:27 PM, Alexander Barkov wrote: > >> Hello Jerome, > >> > >> I have implemented "MDEV-14139 Anchored data types for variables". > >> and pushed it to bb-10.2-ext: > >> > >> > https://github.com/MariaDB/server/commit/5dd5253f7e50c21fa758e2eb58f > 3 > >> aa9c9754e733 > >> > >> So now it should be easier to implement consistent SET by creating > >> backup variables. > >> > >> > >> LEX::sp_variable_declarations_vartype_finalize() implements the logic > >> which copies data type from another variable. > >> > >> > >> The idea is that for all variables, which are assignment targets in a > >> SET or a SELECT INTO statement, we create a backup variable. > >> > >> It will involve these calls for every such variable: > >> > >> LEX::sp_variable_declarations_init(thd, 1); > >> sp_pcontext::add_variable(thd, backup_variable_name); > >> LEX::sp_variable_declarations_vartype_finalize(thd, 1, > >> orig_variable_name, def); > >> > >> where "def" is Item_splocal created for the original variable. > > > > Just an idea: instead of creating sp_instr_set, it's easier to > > introduce a new class sp_instr_set_var, to copy the value from one > > variable to another variable. > > > > This operation will not need neither Item, nor sp_lex_keeper. It will > > only need two offsets: > > - the source variable offset and > > - the target variable offset. > > > > Using these offsets, we can access to > > spcont->m_var_table->field[source] and > > spcont->spcont->m_var_table->field[target] > > and copy the value between them using Field::store_field(). > > > > This won't however for the ROW variables at the moment, because ROW > > fields are stored in the Item_spvar_args::m_table member of > > Item_field_row. > > > > It seems we need a new class Field_row and move Virtual_tmp_table from > > Item_field_row to Field_row. > > > > I will try to do it. > > > I have implemented "MDEV-14212 Add Field_row for SP ROW variables" and > pushed to bb-10.2-ext. > > Also, added a comment: > > MDEV-13418 Compatibility: The order of evaluation of SELECT..INTO > assignments > > > Now, when MDEV-14212 is done, these declarations: > > DECLARE a_tmp TYPE OF a DEFAULT a; > DECLARE b_tmp TYPE OF b DEFAULT b; > > and these assignments: > > SET a=a_tmp; > SET b=b_tmp; > > can use a new sp_instr_setvar instead of sp_inst_set. > > The new command sp_instr_setvar can do direct copying between two > spcont->m_var_table->field[XXX], without a need to create Item and LEX. > > > > >> > >> Greetings. > >>

2 9

[Maria-developers] Summary of the InnoDB changes in the AliSQL 5.6.32 branch
by Marko Mäkelä 08 Nov '17

08 Nov '17

In preparation for the 2017-2 MariaDB Developer Unconference in Shenzhen <https://mariadb.org/2017-2-developers-unconference-and-related-events-shenz…> next week, I cloned the https://github.com/alibaba/AliSQL repository and reviewed the changes to storage/innobase. The repository is based on commit c3a8ca1ed0063d4bca761ac9b57718796242ccc1 (tag: mysql-5.6.32) in the https://github.com/mysql/mysql-server/ repository. I read the 13,587 lines of diff generated by the following command: git diff c3a8ca1ed0063d4bca761ac9b57718796242ccc1 storage/innobase Features that already exist in MariaDB in some form: * Split the btr_search_latch (already present in MySQL 5.7/MariaDB 10.2.2) * Buffer pool "hazard pointer" (5.7/10.2.2) * Faster read view creation (done differently in WL#6578 MySQL 5.7/MariaDB 10.2.2) * Persistent auto_increment (done differently in MDEV-6076 in MariaDB 10.2.4) * Instant ADD COLUMN (done differently in MDEV-11369 in MariaDB 10.3.2) * Sequence engine (done differently in MDEV-10139 in MariaDB 10.3.1) New features: * Counters for buffer pool LRU eviction: MONITOR_LRU_BATCH are split into MONITOR_LRU_BATCH_FLUSH and MONITOR_LRU_BATCH_EVICT (Sounds like a good idea to me; maybe MariaDB should export these via status variables.) * Issue#24 SPLIT LOG BUFFER TO ROTATE LOG WRITE (I’d go further and change the LSN to count mtr_t::commit() instead of bytes, and to use a global buffer for mtr_t::log. This would require a new redo log format.) * Issue#31 OPTIMIZE CHECK/GRANT OF INNODB TABLE LOCK (I’d rather replace InnoDB table locks and dict_operation_lock with MDL.) * Issue#29: ADD INFORMATION_SCHEMA.INNODB_RSEG TABLE (I’d rather split the undo log into tablename.ibu files, also to ease import/export/backup.) * Issue#9: PROVIDE ADAPTIVE ALGORITHM FOR INNODB CONCURRENCY TICKETS (I think that the whole ticket business should be moved to the thread pool plugin.) * Issue#34: Support force drop table (I’d rather remove the InnoDB dictionary and have transactional .frm files, instead of writing work-arounds.) Best regards, Marko -- Marko Mäkelä, Lead Developer InnoDB MariaDB Corporation

1 0

Re: [Maria-developers] d78273d: MDEV-14288 Add MTR option to run tests with real server defaults
by Sergei Golubchik 05 Nov '17

05 Nov '17

Hi, Elena! On Nov 05, elenst(a)montyprogram.com wrote: > revision-id: d78273d1e53308c8fe78b9e067a88c43bd05e540 > parent(s): 737a4da94ae9b25a7ebe828c5b35a1ce4acf03d6 > committer: Elena Stepanova > branch nick: bb-10.0-elenst > timestamp: 2017-11-05 01:30:40 +0200 > message: > > MDEV-14288 Add MTR option to run tests with real server defaults > > diff --git a/mysql-test/lib/My/ConfigFactory.pm b/mysql-test/lib/My/ConfigFactory.pm > index b8ce038..dcdb534 100644 > --- a/mysql-test/lib/My/ConfigFactory.pm > +++ b/mysql-test/lib/My/ConfigFactory.pm > @@ -37,7 +37,9 @@ sub add_opt_values { > > # add auto-options > $config->insert('OPT', 'port' => sub { fix_port($self, $config) }); > - $config->insert('mysqld', "loose-skip-plugin-$_" => undef) for (@::optional_plugins); > + if (!$::opt_default_server_options) { > + $config->insert('mysqld', "loose-skip-plugin-$_" => undef) for (@::optional_plugins); > + } Dunno. I'd prefer to remove magic from mtr, not to add it. Generating different configs feels like magic. I wonder whether this could work: - mtr always disables plugins, but in a separate .cnf file. And default_mysqld.cnf will !include it, but yours won't. or - all this disabling will go where, say, @DISABLE_ALL_PLUGINS line is. it'll be in default_mysqld.cnf but not in yours. > } > > my @pre_rules= > diff --git a/mysql-test/lib/mtr_cases.pm b/mysql-test/lib/mtr_cases.pm > index 38c52b7..26a4edd 100644 > --- a/mysql-test/lib/mtr_cases.pm > +++ b/mysql-test/lib/mtr_cases.pm > @@ -876,7 +876,10 @@ sub collect_one_test_case { > if (not $config) > { > # Suite has no config, autodetect which one to use > - if ($tinfo->{rpl_test}) { > + if ($::opt_default_server_options) { > + $config= "include/hard_default_my.cnf"; > + } > + elsif ($tinfo->{rpl_test}) { yuck. I never liked that rpl suite magically gets a different cnf file. may be it's time to fix it? e.g. by moving it into suite/rpl/my.cnf or into include/rpl_init.cnf ? > $config= "suite/rpl/my.cnf"; > } else { > $config= "include/default_my.cnf"; > diff --git a/mysql-test/mysql-test-run.pl b/mysql-test/mysql-test-run.pl > index 33d9c7b..997fb82 100755 > --- a/mysql-test/mysql-test-run.pl > +++ b/mysql-test/mysql-test-run.pl > @@ -131,6 +131,7 @@ our $plugindir; > my $path_vardir_trace; # unix formatted opt_vardir for trace files > my $opt_tmpdir; # Path to use for tmp/ dir > my $opt_tmpdir_pid; > +our $opt_default_server_options; > > my $opt_start; > my $opt_start_dirty; > @@ -1106,6 +1107,9 @@ sub command_line_setup { > # Extra config file to append to all generated configs > 'defaults-extra-file=s' => \&collect_option, > > + # Use hard defaults instead of heavily modified MTR configuration > + 'default-server-options' => \$opt_default_server_options, with the changes above, you could use --defaults-file=hard_default_my or something like that (even better with a short cnf file name) You could even commit a bunch of templates, like vanilla.cnf, rpm.cnf, debian.cnf and use them like ./mtr --defaults-file=debian > + > # Control what test suites or cases to run > 'force+' => \$opt_force, > 'with-ndbcluster-only' => \&collect_option, > @@ -3509,12 +3517,14 @@ sub mysql_install_db { > mtr_add_arg($args, "--basedir=%s", $install_basedir); > mtr_add_arg($args, "--datadir=%s", $install_datadir); > mtr_add_arg($args, "--plugin-dir=%s", $plugindir); > - mtr_add_arg($args, "--default-storage-engine=myisam"); > - mtr_add_arg($args, "--loose-skip-plugin-$_") for @optional_plugins; > - # starting from 10.0 bootstrap scripts require InnoDB > - mtr_add_arg($args, "--loose-innodb"); > - mtr_add_arg($args, "--loose-innodb-log-file-size=5M"); > - mtr_add_arg($args, "--disable-sync-frm"); > + if (!$opt_default_server_options) { > + mtr_add_arg($args, "--default-storage-engine=myisam"); > + mtr_add_arg($args, "--loose-skip-plugin-$_") for @optional_plugins; > + # starting from 10.0 bootstrap scripts require InnoDB > + mtr_add_arg($args, "--loose-innodb"); > + mtr_add_arg($args, "--loose-innodb-log-file-size=5M"); > + mtr_add_arg($args, "--disable-sync-frm"); > + } why is that? it's just mysql_install_db > mtr_add_arg($args, "--tmpdir=%s", "$opt_vardir/tmp/"); > mtr_add_arg($args, "--core-file"); > mtr_add_arg($args, "--console"); Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] 58c9df453f5: Remove usage of unknown parameter.
by Marko Mäkelä 02 Nov '17

02 Nov '17

This file should not have been re-added in the first place. I removed this badly working file for a good reason. Please try to read the git history before re-adding broken tests. We now have many useless and outright broken tests in --suite=innodb_gis that need some cleanup. I would have liked to review the test changes. Good tests are as valuable as good code. commit 86e0a73eaa166f752d62b31e96925e29c4fe0c8c Author: Marko Mäkelä <marko.makela(a)mariadb.com> Date: Mon Aug 7 23:38:27 2017 +0300 Remove wait_innodb_all_purged.inc The file wait_innodb_all_purged.inc waited for InnoDB purge in a way that only worked in debug builds. The file wait_all_purged.inc provides a better mechanism. The wait_all_purged.inc was added almost 6 months ago: file commit 0bfa3dff8b95ed8b8bd133e434b47189857121ac Author: Marko Mäkelä <marko.makela(a)mariadb.com> Date: Tue May 16 12:07:26 2017 +0300 MDEV-12698 innodb.innodb_stats_del_mark test failure On Tue, Oct 24, 2017 at 12:24 PM, jan <jan.lindstrom(a)mariadb.com> wrote: > revision-id: 58c9df453f5d8408dd189fb2e5ae4445446b6701 (mariadb-10.2.9-80-g58c9df453f5) > parent(s): 39f86dd27a4c772f0b0a65fbf9b125be9e2dc8ac > author: Jan Lindström > committer: Jan Lindström > timestamp: 2017-10-24 12:23:57 +0300 > message: > > Remove usage of unknown parameter. > > --- > mysql-test/include/wait_innodb_all_purged.inc | 1 - > 1 file changed, 1 deletion(-) > > diff --git a/mysql-test/include/wait_innodb_all_purged.inc b/mysql-test/include/wait_innodb_all_purged.inc > index 5bf816fd72f..c5edc5937da 100644 > --- a/mysql-test/include/wait_innodb_all_purged.inc > +++ b/mysql-test/include/wait_innodb_all_purged.inc > @@ -48,7 +48,6 @@ WHERE VARIABLE_NAME = 'INNODB_PURGE_TRX_ID_AGE';`; > } > if (!$success) > { > - set global innodb_purge_run_now=ON; > real_sleep 0.1; > dec $wait_counter; > } > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- Marko Mäkelä, Lead Developer InnoDB MariaDB Corporation

1 0

[Maria-developers] 10.2.10 compiler warnings Fedora 26 / GCC7
by Reindl Harald 01 Nov '17

01 Nov '17

maybe worth to take a look [ 8%] Building C object libmariadb/mariadb_config/CMakeFiles/mariadb_config.dir/mariadb_config.c.o /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/libmariadb/mariadb_config/mariadb_config.c:9:49: warning: backslash and newline separated by space #define LIBS "-L/usr/lib64/mysql/ -lmariadb "\ [ 20%] Building C object libmariadb/libmariadb/CMakeFiles/mariadb_obj.dir/ma_dtoa.c.o /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/libmariadb/libmariadb/ma_dtoa.c: In function 'dtoa': /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/libmariadb/libmariadb/ma_dtoa.c:1465:14: warning: this statement may fall through [-Wimplicit-fallthrough=] leftright= 0; ~~~~~~~~~^~~ /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/libmariadb/libmariadb/ma_dtoa.c:1467:3: note: here case 4: ^~~~ /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/libmariadb/libmariadb/ma_dtoa.c:1473:14: warning: this statement may fall through [-Wimplicit-fallthrough=] leftright= 0; ~~~~~~~~~^~~ /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/libmariadb/libmariadb/ma_dtoa.c:1475:3: note: here case 5: ^~~~ At top level: /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/libmariadb/libmariadb/ma_dtoa.c:1161:21: warning: 'tinytens' defined but not used [-Wunused-const-variable=] static const double tinytens[]= ^~~~~~~~ [ 24%] Building C object strings/CMakeFiles/strings.dir/dtoa.c.o /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/libmariadb/libmariadb/mariadb_dyncol.c: In function 'mariadb_dyncol_val_double': /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/libmariadb/libmariadb/mariadb_dyncol.c:4049:15: warning: argument 1 null where non-null expected [-Wnonnull] *dbl= strtod(str, &end); ^~~~~~~~~~~~~~~~~ In file included from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/libmariadb/include/ma_global.h:168:0, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/libmariadb/libmariadb/mariadb_dyncol.c:64: /usr/include/stdlib.h:126:15: note: in a call to function 'strtod' declared here extern double strtod (const char *__restrict __nptr, [ 33%] Building C object mysys/CMakeFiles/mysys.dir/my_malloc.c.o /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/mysys/my_lib.c: In function 'my_dir': /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/mysys/my_lib.c:144:3: warning: 'readdir_r' is deprecated [-Wdeprecated-declarations] while (!(READDIR(dirp,(struct dirent*) dirent_tmp,dp))) ^~~~~ In file included from /usr/include/features.h:410:0, from /usr/include/pthread.h:21, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/include/my_global.h:278, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/mysys/mysys_priv.h:19, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/mysys/my_lib.c:19: /usr/include/dirent.h:189:12: note: declared here extern int __REDIRECT (readdir_r, [ 68%] Building C object storage/maria/CMakeFiles/aria.dir/ma_check.c.o /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/storage/myisam/myisamchk.c: In function 'myisamchk': /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/storage/myisam/myisamchk.c:1411:22: warning: '%ld' directive writing between 1 and 10 bytes into a region of size 8 [-Wformat-overflow=] sprintf(null_pos,"%ld",(long) keyseg->null_pos+1); ^~~ /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/storage/myisam/myisamchk.c:1411:21: note: directive argument in the range [1, 4294967296] sprintf(null_pos,"%ld",(long) keyseg->null_pos+1); ^~~~~ In file included from /usr/include/stdio.h:939:0, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/include/my_global.h:322, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/include/my_base.h:23, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/include/myisam.h:26, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/storage/myisam/myisamdef.h:20, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/storage/myisam/fulltext.h:21, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/storage/myisam/myisamchk.c:18: /usr/include/bits/stdio2.h:33:10: note: '__builtin___sprintf_chk' output between 2 and 11 bytes into a destination of size 8 return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ __bos (__s), __fmt, __va_arg_pack ()); ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [ 68%] Building C object storage/maria/CMakeFiles/aria.dir/ma_preload.c.o /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/storage/myisam/mi_test3.c: In function 'test_write': /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/storage/myisam/mi_test3.c:367:30: warning: '%7ld' directive writing between 7 and 11 bytes into a region of size 8 [-Wformat-overflow=] sprintf((char*) record.id,"%7ld",(long) getpid()); ^~~~ /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/storage/myisam/mi_test3.c:367:29: note: directive argument in the range [-2147483648, 2147483647] sprintf((char*) record.id,"%7ld",(long) getpid()); ^~~~~~ In file included from /usr/include/stdio.h:939:0, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/include/my_global.h:322, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/include/my_base.h:23, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/include/myisam.h:26, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/storage/myisam/mi_test3.c:20: /usr/include/bits/stdio2.h:33:10: note: '__builtin___sprintf_chk' output between 8 and 12 bytes into a destination of size 8 return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ __bos (__s), __fmt, __va_arg_pack ()); ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [ 71%] Building C object storage/maria/CMakeFiles/aria_chk.dir/maria_chk.c.o /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/storage/maria/maria_chk.c: In function 'maria_chk': /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/storage/maria/maria_chk.c:1703:22: warning: '%ld' directive writing between 1 and 10 bytes into a region of size 8 [-Wformat-overflow=] sprintf(null_pos,"%ld",(long) keyseg->null_pos+1); ^~~ /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/storage/maria/maria_chk.c:1703:21: note: directive argument in the range [1, 4294967296] sprintf(null_pos,"%ld",(long) keyseg->null_pos+1); ^~~~~ In file included from /usr/include/stdio.h:939:0, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/include/my_global.h:322, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/include/my_base.h:23, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/include/maria.h:22, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/storage/maria/maria_def.h:18, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/storage/maria/ma_fulltext.h:20, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/storage/maria/maria_chk.c:18: /usr/include/bits/stdio2.h:33:10: note: '__builtin___sprintf_chk' output between 2 and 11 bytes into a destination of size 8 return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ __bos (__s), __fmt, __va_arg_pack ()); ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [ 73%] Building C object storage/maria/CMakeFiles/ma_test3.dir/ma_test3.c.o /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/storage/maria/ma_test3.c: In function 'test_write': /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/storage/maria/ma_test3.c:365:30: warning: '%7ld' directive writing between 7 and 11 bytes into a region of size 8 [-Wformat-overflow=] sprintf((char*) record.id,"%7ld", (long) getpid()); ^~~~ /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/storage/maria/ma_test3.c:365:29: note: directive argument in the range [-2147483648, 2147483647] sprintf((char*) record.id,"%7ld", (long) getpid()); ^~~~~~ In file included from /usr/include/stdio.h:939:0, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/include/my_global.h:322, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/include/my_base.h:23, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/include/maria.h:22, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/storage/maria/maria_def.h:18, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.10/storage/maria/ma_test3.c:20: /usr/include/bits/stdio2.h:33:10: note: '__builtin___sprintf_chk' output between 8 and 12 bytes into a destination of size 8 return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ __bos (__s), __fmt, __va_arg_pack ()); ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

1 0

Re: [Maria-developers] [Commits] f88fcf2: MDEV-12542 Add bind_address system variable.
by Sergei Golubchik 30 Oct '17

30 Oct '17

Hi, Alexey! On Oct 30, Alexey Botchkov wrote: > revision-id: f88fcf2514c64df8408e9df41dc129ab6f655c18 (mariadb-10.3.2-17-gf88fcf2) > parent(s): ecee3c71e1e74039e75089f1d76c0b6eea9e43ba > committer: Alexey Botchkov > timestamp: 2017-10-30 07:37:25 +0400 > message: > > MDEV-12542 Add bind_address system variable. > > bind_address variable added. > > --- > .../suite/sys_vars/r/bind_address_basic.result | 57 +++++++++++++ > .../suite/sys_vars/t/bind_address_basic.test | 95 ++++++++++++++++++++++ There's no need to add these files anymore (since 10.1) They were supposed to test basic properties like num/string/readonly, etc. But now sysvars_server_notembedded.result has that for all variables anyway. So these *_basic tests are not needed. > sql/mysqld.cc | 3 - > sql/mysqld.h | 1 + > sql/sys_vars.cc | 7 ++ > 6 files changed, 174 insertions(+), 3 deletions(-) > Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MDEV-13997 Change Item_bool_rowready_func2 to cache const items at fix time rather than evaluation time
by Alexander Barkov 29 Oct '17

29 Oct '17

Hello Sergei, Please review a patch for MDEV-13997. It's needed for: MDEV-13995 MAX(timestamp) returns a wrong result near DST change and for: MDEV-4912 Add a plugin to field types (column types) Thanks!

2 3

[Maria-developers] Review Custom Aggregate Functions
by Vicențiu Ciorbaru 28 Oct '17

28 Oct '17

Hi Varun! Here's a final review for Custom Aggregate Functions * You haven't run the full test suite, there are a few failing tests. Please update those. rpl.rpl_sp funcs_1.is_columns_mysql funcs_1.storedproc * Push your code to a BB branch and rebase it on top of current 10.3. * Fix commits messages while rebasing. Use the reword flag to clean those up. The rule is this: 1. Have one line that doesn't exceed 80 Characters as a Title. 2. Leave one empty line 3. Add the rest of the commit message. Don't exceed 80 Characters per line. > diff --git a/sql/sp_head.h b/sql/sp_head.h > index bb516187a57..d447807bb3b 100644 > --- a/sql/sp_head.h > +++ b/sql/sp_head.h > @@ -334,6 +334,11 @@ class sp_head :private Query_arena, > execute_function(THD *thd, Item **args, uint argcount, Field *return_fld); > > bool Rename this to execute_function please and delete the now dead-code from the current execute_function. Update comments for this function appropriately. Please run all tests again after this and make sure you update results for all the failing ones. > + execute_aggregate_function(THD *thd, Item **args, uint argcount, > + Field *return_fld, sp_rcontext **nctx, > + MEM_ROOT *caller_mem_root); > + > + bool > execute_procedure(THD *thd, List<Item> *args); > > static void > @@ -813,6 +818,9 @@ class sp_head :private Query_arena, > bool > execute(THD *thd, bool merge_da_on_success); > This is no longer used please remove! > + bool > + execute_agg(THD *thd, bool merge_da_on_success); > + > /** > Perform a forward flow analysis in the generated code. > Mark reachable instructions, for the optimizer. > +++ b/mysql-test/t/sp_agg.test A lot of empty lines at the end of this test. Please delete those. Ping me once you have a green buildbot before pushing. Vicentiu

1 0

[Maria-developers] Review of Percentile Window Functions
by Vicențiu Ciorbaru 28 Oct '17

28 Oct '17

Hi Varun, Here's the final review for Percentile Functions. Just 2 minor coding style comments. Rebase on top of current 10.3 and push to a bb-* branch to double check nothing breaks. Make sure that no merge commit happens. Regards, Vicențiu > diff --git a/sql/item_windowfunc.h b/sql/item_windowfunc.h > index 77cbd556e60..589f7cf5d96 100644 > --- a/sql/item_windowfunc.h > +++ b/sql/item_windowfunc.h > @@ -678,6 +687,277 @@ class Item_sum_ntile : public Item_sum_window_with_row_count > ulong current_row_count_; > }; > > +class Item_sum_percentile_disc : public Item_sum_cume_dist, > + public Type_handler_hybrid_field_type > +{ > +public: > + Item_sum_percentile_disc(THD *thd, Item* arg) : Item_sum_cume_dist(thd, arg), > + Type_handler_hybrid_field_type(&type_handler_longlong), > + value(NULL), val_calculated(FALSE), first_call(TRUE), > + prev_value(0), order_item(NULL){} > + > + double val_real() > + { > + if (get_row_count() == 0 || get_arg(0)->is_null()) > + { > + null_value= true; > + return 0; > + } > + null_value= false; > + return value->val_real(); > + } > + > + longlong val_int() > + { > + if (get_row_count() == 0 || get_arg(0)->is_null()) > + { > + null_value= true; > + return 0; > + } > + null_value= false; > + return value->val_int(); > + } > + > + my_decimal* val_decimal(my_decimal* dec) > + { > + if (get_row_count() == 0 || get_arg(0)->is_null()) > + { > + null_value= true; > + return 0; > + } > + null_value= false; > + return value->val_decimal(dec); > + } > + > + String* val_str(String *str) > + { > + if (get_row_count() == 0 || get_arg(0)->is_null()) > + { > + null_value= true; > + return 0; > + } > + null_value= false; > + return value->val_str(str); > + } > + > + bool add() > + { > + Item *arg= get_arg(0); > + if (arg->is_null()) > + return false; > + > + if (first_call) > + { > + prev_value= arg->val_real(); > + if (prev_value > 1 || prev_value < 0) > + { > + my_error(ER_ARGUMENT_OUT_OF_RANGE, MYF(0), func_name()); > + return true; > + } > + first_call= false; > + } > + > + double arg_val= arg->val_real(); > + > + if (prev_value != arg_val) > + { > + my_error(ER_ARGUMENT_NOT_CONSTANT, MYF(0), func_name()); > + return true; > + } > + > + if (val_calculated) > + return false; > + > + value->store(order_item); > + value->cache_value(); > + if (value->null_value) > + return false; > + > + Item_sum_cume_dist::add(); > + double val= Item_sum_cume_dist::val_real(); > + > + if (val >= prev_value && !val_calculated) > + val_calculated= true; > + return false; > + } > + > + enum Sumfunctype sum_func() const > + { > + return PERCENTILE_DISC_FUNC; > + } > + > + void clear() > + { > + val_calculated= false; > + first_call= true; > + value->clear(); > + Item_sum_cume_dist::clear(); > + } > + > + const char*func_name() const > + { > + return "percentile_disc"; > + } > + > + void update_field() {} > + void set_type_handler(Window_spec *window_spec); > + const Type_handler *type_handler() const > + {return Type_handler_hybrid_field_type::type_handler();} > + > + void fix_length_and_dec() > + { > + decimals = 10; // TODO-cvicentiu find out how many decimals the standard > + // requires. > + } > + > + Item *get_copy(THD *thd, MEM_ROOT *mem_root) > + { return get_item_copy<Item_sum_percentile_disc>(thd, mem_root, this); } > + void setup_window_func(THD *thd, Window_spec *window_spec); > + void setup_hybrid(THD *thd, Item *item); > + bool fix_fields(THD *thd, Item **ref); > + > +private: > + Item_cache *value; > + bool val_calculated; > + bool first_call; > + double prev_value; > + Item *order_item; > +}; > + > +class Item_sum_percentile_cont : public Item_sum_cume_dist, > + public Type_handler_hybrid_field_type > +{ > +public: > + Item_sum_percentile_cont(THD *thd, Item* arg) : Item_sum_cume_dist(thd, arg), > + Type_handler_hybrid_field_type(&type_handler_double), > + floor_value(NULL), ceil_value(NULL), first_call(TRUE),prev_value(0), > + ceil_val_calculated(FALSE), floor_val_calculated(FALSE), order_item(NULL){} > + > + double val_real() > + { > + if (get_row_count() == 0 || get_arg(0)->is_null()) > + { > + null_value= true; > + return 0; > + } > + null_value= false; > + double val= 1 + prev_value * (get_row_count()-1); > + > + /* > + Applying the formula to get the value > + If (CRN = FRN = RN) then the result is (value of expression from row at RN) > + Otherwise the result is > + (CRN - RN) * (value of expression for row at FRN) + > + (RN - FRN) * (value of expression for row at CRN) > + */ > + > + if(ceil(val) == floor(val)) > + return floor_value->val_real(); > + > + double ret_val= ((val - floor(val)) * ceil_value->val_real()) + > + ((ceil(val) - val) * floor_value->val_real()); > + > + return ret_val; > + } > + > + bool add() > + { > + Item *arg= get_arg(0); > + if (arg->is_null()) > + return false; > + > + if (first_call) > + { > + first_call= false; > + prev_value= arg->val_real(); > + if (prev_value > 1 || prev_value < 0) > + { > + my_error(ER_ARGUMENT_OUT_OF_RANGE, MYF(0), func_name()); > + return true; > + } > + } > + > + double arg_val= arg->val_real(); > + if (prev_value != arg_val) > + { > + my_error(ER_ARGUMENT_NOT_CONSTANT, MYF(0), func_name()); > + return true; > + } > + > + if (!floor_val_calculated) > + { > + floor_value->store(order_item); > + floor_value->cache_value(); > + if (floor_value->null_value) > + return false; Add one extra space before return > + } > + if (floor_val_calculated && !ceil_val_calculated) > + { > + ceil_value->store(order_item); > + ceil_value->cache_value(); > + if (ceil_value->null_value) > + return false; Add one extra space before return > + } > + > + Item_sum_cume_dist::add(); > + double val= 1 + prev_value * (get_row_count()-1); > + > + if (!floor_val_calculated && get_row_number() == floor(val)) > + floor_val_calculated= true; too much indentation here. Delete 2 spaces > + > + if (!ceil_val_calculated && get_row_number() == ceil(val)) > + ceil_val_calculated= true; too much indentation here. Delete 2 spaces > + return false; > + } > + > + enum Sumfunctype sum_func() const > + { > + return PERCENTILE_CONT_FUNC; > + } > + > + void clear() > + { > + first_call= true; > + floor_value->clear(); > + ceil_value->clear(); > + floor_val_calculated= false; > + ceil_val_calculated= false; > + Item_sum_cume_dist::clear(); > + } > + > + const char*func_name() const > + { > + return "percentile_cont"; > + } > + void update_field() {} > + void set_type_handler(Window_spec *window_spec); > + const Type_handler *type_handler() const > + {return Type_handler_hybrid_field_type::type_handler();} > + > + void fix_length_and_dec() > + { > + decimals = 10; // TODO-cvicentiu find out how many decimals the standard > + // requires. > + } > + > + Item *get_copy(THD *thd, MEM_ROOT *mem_root) > + { return get_item_copy<Item_sum_percentile_cont>(thd, mem_root, this); } > + void setup_window_func(THD *thd, Window_spec *window_spec); > + void setup_hybrid(THD *thd, Item *item); > + bool fix_fields(THD *thd, Item **ref); > + > +private: > + Item_cache *floor_value; > + Item_cache *ceil_value; > + bool first_call; > + double prev_value; > + bool ceil_val_calculated; > + bool floor_val_calculated; > + Item *order_item; > +}; > + > + > + > > class Item_window_func : public Item_func_or_sum > { > diff --git a/sql/lex.h b/sql/lex.h > index 17dd45f9c0b..fcb44a0f836 100644 > --- a/sql/lex.h > +++ b/sql/lex.h > @@ -735,6 +736,7 @@ static SYMBOL sql_functions[] = { > { "LAG", SYM(LAG_SYM)}, > { "LEAD", SYM(LEAD_SYM)}, > { "MAX", SYM(MAX_SYM)}, > + { "MEDIAN", SYM(MEDIAN_SYM)}, Fix indentation here > { "MID", SYM(SUBSTRING)}, /* unireg function */ > { "MIN", SYM(MIN_SYM)}, > { "NOW", SYM(NOW_SYM)}, > \ No newline at end of file

1 0

Re: [Maria-developers] 9145b7d: MDEV-11153 - Introduce status variables for table cache monitoring and tuning
by Sergei Golubchik 27 Oct '17

27 Oct '17

Hi, Sergey! Looks ok, thanks. Just one question: On Aug 10, Sergey Vojtovich wrote: > revision-id: 9145b7dd11a0e95016304e53acf306daac323580 (mariadb-10.3.0-49-g9145b7d) > parent(s): 63ad4fe5bbe77fe17ab441fcce0106bdee9b618d > committer: Sergey Vojtovich > timestamp: 2017-08-10 15:45:03 +0400 > message: > > MDEV-11153 - Introduce status variables for table cache monitoring and tuning > > Status variables added: Table_open_cache_hits, Table_open_cache_misses, > Table_open_cache_overflows. > > System variables added: table_open_cache_active_instances (read-only). > > diff --git a/sql/sys_vars.cc b/sql/sys_vars.cc > index e89dd47..92721b6 100644 > --- a/sql/sys_vars.cc > +++ b/sql/sys_vars.cc > @@ -3219,6 +3219,12 @@ static Sys_var_ulong Sys_table_cache_size( > BLOCK_SIZE(1), NO_MUTEX_GUARD, NOT_IN_BINLOG, ON_CHECK(0), > ON_UPDATE(fix_table_open_cache)); > > +static Sys_var_uint Sys_table_cache_active_instances( > + "table_open_cache_active_instances", > + "Number of active table cache instances", > + READ_ONLY GLOBAL_VAR(tc_active_instances), NO_CMD_LINE, > + VALID_RANGE(1, 64), DEFAULT(1), BLOCK_SIZE(1)); Why did you do it as a sysvar, not as a status variable? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 2

[Maria-developers] Code syntax: questions on pointers, etc.
by Aleksey Midenkov 27 Oct '17

27 Oct '17

Recent refactorings of replacing C strings with LEX_CSTRING which is no doubt a good thing raise some questions: 1. Is it still guaranteed that Field::field_name.str is NULL-terminated? 2. It is still passed as a pointer to functions. Why is that? The main feature of C++ references is that it cannot be NULL, so we get segfault on top of the stack (closer to a cause), not the bottom of it. I see that pointers are now widely used and mainly assumed to be always non-NULL (i.e. dereferenced without assertion). But placing such implicit contract on data is not evident and bug-prone. IMHO it's much better to use references whenever it is possible (and when there is no need in cosy NULL semantic). What do you think? But for such lightweight structs like LEX_CSTRING it is even better to pass by value, so we could have the conventience of type cast. 3. LEX_CSTRING and LEX_STRING are now non-convertible. Why not to make: template <typename char_t> struct st_mysql_lex_string { char_t *str; size_t length; }; typedef st_mysql_lex_string<char *> LEX_STRING; typedef st_mysql_lex_string<const char *> LEX_CSTRING; typedef st_mysql_lex_string<const unsigned char *> LEX_CUSTRING; ? 4. There are some duplicate types: MYSQL_LEX_STRING, MYSQL_CONST_LEX_STRING. Why? -- All the best, Aleksey Midenkov @midenok

2 3

[Maria-developers] Obsolete GTID domain delete on master (MDEV-12012, MDEV-11969)
by andrei.elkin＠pp.inet.fi 24 Oct '17

24 Oct '17

Hello. Let me propose methods to clean master off unused gtid domains. The issue is quite practical, as a couple of references on the subject line tell. Either of them narrates a scenario of two default domain-id masters serving to one slave initially by "legacy" non-gtid protocol. When later the masters have changed their common domain to different private ones *and* the slave turns gtid replication ON, it can't connect to a master due to the gtid protocol. And the reason is the prior-to-gtid-connect gained slave's gtid position consisting of gtids generated by the other master obviously (the two masters never replicated each other) does not fit to the current master binlog state. The past default domain-id is actually permanent past from the user perspective in these cases. Its events have been already replicated and none new will be generated and replicated. Therefore such domain conceptually may be cleaned away from either the masters and slave states. Once it's done, the gtid-enabled slave will successfully connect to any master. The slave state purge is simple SET @@global.gtid_slave_pos to a sequence free of the purged domain. The master side binlog state one requires a new "command" unless the user is happy with RESET MASTER. While setting the new gtid binlog state to be old-domain-free we would like for the new "command" to preserve the existing binlog files. This could be accomplished as Kristian suggests in MDEV-12012 (I could not find any earlier references to the idea) as a new option to FLUSH BINARY LOGS DELETE DOMAIN d1, d2 KN> This command would check that the current binlog state is equal to KN> the GTID_LIST_LOG_EVENT at the start of the first binary log file, KN> within the specified domains. If not, it is an error. But if so, KN> the new binary log file created by FLUSH will be written with the KN> specified domains omitted from GTID_LIST_LOG_EVENT (and the current KN> binlog state updated accordingly). The idea looks quite sane, I only could not grasp why presence of being deleted domains in the very first binlog's GTID_LIST_LOG_EVENT list is warrant for throwing error. Maybe we should leave it out to the user, Kristian? That is to decide what domain is garbage regardless of the binlog state history. While the FLUSH way looks sufficient and robust I could not help to think over an alternative. Consider a scenario when a domain's sequence number got run out of range. While deems unrealistic in practice we can simulate it with SET @@SESSION.gtid_domain_id=11; SET @@SESSION.server_id=1; SET @@SESSION.gtid_seq_no=18446744073709551615; /* Exec some dummy loggable query, e.g */ CREATE TABLE IF NOT EXISTS `table_dummy`; SHOW LOCAL VARIABLES LIKE '%gtid_binlog_pos%'; 11-1-18446744073709551615 SET @@SESSION.gtid_seq_no=0 DROP TABLE `table_dummy`; SHOW LOCAL VARIABLES LIKE '%gtid_binlog_pos%'; 11-1-0 I've used two gtids to show domain overflow because I also liked to read the zero of the last gtid as ... there's *nothing* in this domain. So it's actually a new "namesake" one, replacing the old that is wrapped around. The 1st group of events created in the new domain - 11-1-1 - could shadow the old domain's 11-1-1 as well as all the rest of the old domain from gtid replication protocol. And that means the old domain is actually deleted. So if my reading of zero is correct the binlog status would be empty instead. That's how we also can approach the master side old gtid domain purging: 1. Leave wrapping around an old domain to the user via running the queries like above; 2. The binary logger would be made to react on the fact of wrap-around with binary log rotation ("internal" FLUSH BINARY LOG). And the new binlog file won't contain the wrapped "away" domain (because there are no new event group in it of yet). I would be glad to hear your opinions, dear colleagues. Cheers, Andrei

4 27

[Maria-developers] Bitmap indexing with Mariadb
by chanaka rathnayaka 16 Oct '17

16 Oct '17

Hi, I am Chanaka who is an undergraduate engineering student. I am doing a research to check, how bitmap indexing improves the query performance and I hope to use MariaDB as database engine. So I need to know whether MariaDB has bitmap indexing already or not ? Thank you, Regards, Chanaka <http://www.justforlearning.info> Chanaka Rathnayaka / undergraduate, Department of Computer Science and Engineering, University of Moratuwa, Sri Lanka bmcrathnayaka(a)gmail.com / +94776608457 [image: Twitter] <https://htmlsig.com/t/0000001BX4MCF> [image: Facebook] <https://htmlsig.com/t/0000001BR3W18> [image: LinkedIn] <https://htmlsig.com/t/0000001BT4W90> [image: Github] <https://htmlsig.com/t/0000001BYB6WK>

2 1

[Maria-developers] MDEV-13550 is test case required?
by Aleksey Midenkov 14 Oct '17

14 Oct '17

Hello! Do we have to write test cases for such coding errors? Because this is developer factor: either he does such errors or not. What are conditions of getting PR merged?

2 2

Re: [Maria-developers] MDEV-14013
by Alexander Barkov 14 Oct '17

14 Oct '17

Hello Jerome, Thank you very much for you contribution! On 10/11/2017 02:32 PM, jerome brauge wrote: > Hello Alexander, > Here is the patch for MDEV-14013, can you review it ? The patch looks very good. I only found small things: 1. LEX::make_text_literal_concat() thd->variables.character_set_client was used instread of thd->variables.collation_connection 2. LEX::make_text_literal_concat() was somewhat hard to read. I propose to change the logic to do: if (item->type() == Item::NULL_ITEM) { } instead of if (item->type() != Item::NULL_ITEM || !(thd->variables.sql_mode & MODE_EMPTY_STRING_IS_NULL)) I think it looks clearer this way. Also, I added some DBUG_ASSERTs to make more sure that a non-Item_string item is not erroneously cast to Item_string. 3. Also, I found that it's easier to place the new methods into THD instead of LEX. Please find a patch with changes 1,2,3 attached. Do you agree with these changes? Also, can you please add some tests: - SHOW CREATE VIEW into "Test in a view" in empty_string_literal.result.diff - EXPLAIN EXTENDED for the affected grammar: EXPLAIN EXTENDED SELECT ''; EXPLAIN EXTENDED SELECT _latin1''; EXPLAIN EXTENDED SELECT N''; EXPLAIN EXTENDED SELECT '' ''; Thanks! > > Thanks. > > >> -----Message d'origine----- >> De : Alexander Barkov [mailto:bar@mariadb.org] >> Envoyé : mercredi 11 octobre 2017 04:15 >> À : jerome brauge >> Objet : Re: Question about MDEV-10574 >> >> Hello Jerome, >> >> >> On 10/10/2017 04:23 PM, jerome brauge wrote: >>> Hello Alexander, >>> For sql_mode EMPTY_STRING_IS_NULL, do you think that it must cover call >> of statement with use of user variable initialized with an empty string ? >>> >>> Ex : >>> SET sql_mode=default; >>> SET @param=''; >>> >>> SET sql_mode=EMPTY_STRING_IS_NULL; >>> PREPARE test FROM 'select 1,?'; >>> EXECUTE test USING @param; >>> >>> Or >>> >>> delimiter / >>> CREATE OR REPLACE PROCEDURE p1(IN a VARCHAR(50)) BEGIN >>> IF a is null THEN >>> SELECT 'IS NULL'; >>> ELSE >>> select 'IS NOT NULL'; >>> END IF; >>> END; >>> / >>> delimiter ; >>> SET sql_mode=default; >>> SET @a=''; >>> >>> SET sql_mode=EMPTY_STRING_IS_NULL; >>> EXECUTE IMMEDIATE 'CALL p1(?)' USING @a; CALL p1(@a); >>> >>> For me, it's a study case and we can forget it. >> >> I agree. Oracle doesn't have user variables. >> >>> >>> Regards, >>> Jérôme. >>> >>>> -----Message d'origine----- >>>> De : Alexander Barkov [mailto:bar@mariadb.org] Envoyé : samedi 7 >>>> octobre 2017 11:44 À : jerome brauge Objet : Re: Patch for MDEV-10574 >>>> >>>> Hello Jerome, >>>> >>>> >>>> On 10/06/2017 01:41 PM, jerome brauge wrote: >>>>> Hello Alexander, >>>>> >>>>>> -----Message d'origine----- >>>>>> De : Alexander Barkov [mailto:bar@mariadb.org] Envoyé : jeudi 5 >>>>>> octobre 2017 21:26 À : jerome brauge Objet : Re: Patch for >>>>>> MDEV-10574 >>>>>> >>>>>> Hello Jerome, >>>>>> >>>>>> >>>>>> On 10/05/2017 12:45 PM, jerome brauge wrote: >>>>>>> Hello Alexander, >>>>>>> Have you gone ahead on your reflection about nulls and empty strings >> ? >>>>>> >>>>>> Yes, I think so. >>>>>> >>>>>> One addition: Monty suggested to rename flags to: >>>>>> - EMPTY_STRING_IS_NULL for literals >>>>>> - EMPTY_STRING_IS_NULL_RESULTS for functions. >>>>>> >>>>>> I'm fine with EMPTY_STRING_IS_NULL for literals, but think that >>>>>> EMPTY_STRING_IS_NULL_FUNCTIONS should be more clear than >>>>>> EMPTY_STRING_IS_NULL_RESULTS. >>>>>> Which name do you prefer for functions? >>>>> >>>>> I think like you. >>>>> For me, EMPTY_STRING_IS_NULL_RESULTS sounds like including >>>> RESULTSET >>>>> of select (which is perhaps included in point 4) >>>> >>>> Okey, let's go with _FUNCTIONS then. >>>> >>>>> >>>>>>> >>>>>>> Can I >>>>>>> - redo my pull request for substring_oracle function >>>>>>> - make a patch for first new sql_mode >>>>>> (MODE_EMPTY_AS_NULL_LITERALS) >>>>>> >>>>>> >>>>>> Yes, please. Let's move forward. >>>>>> >>>>>> Let's push the patch for substr_oracle first. >>>>>> Let's keep it return empty strings for the cases when the >>>>>> substring_length parameters is less than 1. >>>>>> It will start automatically returning NULL when we add the new >>>>>> sql_mode for functions. >>>>> >>>>> Pull request is done. >>>> >>>> I have merged it. Thanks! >>>> >>>>> >>>>>> Also, let's go ahead with the part for literals. >>>>> >>>>> Ok, I will make a separate patch. >>>>> Many thanks. >>>>> >>>>>> >>>>>> >>>>>> I have created MDEVs. Please use them in the commit comment and in >>>>>> the >>>>>> tests: >>>>>> >>>>>> MDEV-14012 sql_mode=Oracle: substr(): treat position 0 as position >>>>>> 1 >>>>>> MDEV-14013 sql_mode=EMPTY_STRING_IS_NULL >>>>>> >>>>>> >>>>>> Thanks! >>>>>> >>>>>>> >>>>>>> Regards, >>>>>>> Jérôme. >>>>>>> >>>>>>> >>>>>>>> -----Message d'origine----- >>>>>>>> De : Alexander Barkov [mailto:bar@mariadb.org] Envoyé : mercredi >>>>>>>> 27 septembre 2017 08:50 À : jerome brauge Objet : Re: Patch for >>>>>>>> MDEV-10574 >>>>>>>> >>>>>>>> Hello Jerome, >>>>>>>> >>>>>>>> I told to Monty yesterday. >>>>>>>> >>>>>>>> We were going to implement the following transformations: >>>>>>>> >>>>>>>>> We are considering to implement both of the following options >>>>>>>>> (with a >>>>>>>>> switch) >>>>>>>>> >>>>>>>>> Transform Insert >>>>>>>>> - insert values ("") -> insert values (null) >>>>>>>>> >>>>>>>>> Transform Select >>>>>>>>> >>>>>>>>> where v=x => (v <> "" and V=X) >>>>>>>>> where v is null => (v="" or v is null) >>>>>>>>> >>>>>>>> >>>>>>>> See MDEV-10574 for details. >>>>>>>> >>>>>>>> Monty did not fully understand the idea of translating literals >>>>>>>> and >>>>>> functions >>>>>>>> from empty strings to null. He asked if you can explain use cases >>>>>>>> for these transformations. >>>>>>>> >>>>>>>> But we agreed that: >>>>>>>> >>>>>>>> 1. It's OK to have multiple NULL handling flags in sql_mode. >>>>>>>> >>>>>>>> 2. New flags will not be enabled in 10.3, neither in >>>>>>>> sql_mode=DEFAULT, >>>>>> nor in >>>>>>>> sql_mode=ORACLE. So one will have to specify them explicitly: >>>>>>>> set >>>>>>>> >>>>>> >>>> >> sql_mode='ORACLE,MODE_null_transfrormation_flag1,MODE_null_transfor >>>>>>>> mation_flag2'; >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> So I propose we introduce the following flags: >>>>>>>> >>>>>>>> >>>>>>>> 1. MODE_EMPTY_AS_NULL_LITERALS - for literals and PS >> parameters, >>>> as >>>>>> in >>>>>>>> your patch. >>>>>>>> >>>>>>>> 2. MODE_EMPTY_AS_NULL_FUNCTIONS - for functions. >>>>>>>> This should be done as a separate patch. >>>>>>>> Note, your patch currently implements empty-to-null translation >>>>>>>> only for Item_str_func descendants that use >> make_empty_result(). >>>>>>>> This patch should be extended to cover Item_str_func descendants >>>>>>>> which do not use make_empty_result(), as well as >>>>>>>> all other Item_func_or_sum descendants >>>>>>>> (which are not Item_str_func descendants). >>>>>>>> >>>>>>>> >>>>>>>> 3. MODE_EMPTY_AS_NULL_UPDATES >>>>>>>> >>>>>>>> For INSERT and UPDATE transformation, as in MDEV: >>>>>>>> >>>>>>>> insert values ("") -> insert values (null) >>>>>>>> update set a='' -> update set a=null; >>>>>>>> >>>>>>>> >>>>>>>> 4. MODE_EMPTY_AS_NULL_PREDICATES >>>>>>>> >>>>>>>> For predicate transformation, as in MDEV: >>>>>>>> >>>>>>>> where v=x => (v <> "" and V=X) >>>>>>>> where v is null => (v="" or v is null) >>>>>>>> >>>>>>>> >>>>>>>> What do you think about this proposal with multiple flags? >>>>>>>> Note, we can do N1 and N2 now, and add N3 and N4 and later. >>>>>>>> >>>>>>>> Also, can you please write a description why you need N1 and N2. >>>>>>>> Monty thinks that N3 and N4 should cover most use cases. >>>>>>>> So we need a good rationale for Monty and Sergei explaining why >>>>>>>> we >>>>>> adding >>>>>>>> these flags N1 and N2. >>>>>>>> >>>>>>>> >>>>>>>> Thanks! >>>>>>>> >>>>>>>> >>>>>>>> On 09/26/2017 04:33 PM, jerome brauge wrote: >>>>>>>>> Hi Alexander, >>>>>>>>> I'm disappointed. >>>>>>>>> Can we at least keep this part in sql_prepare : >>>>>>>>> >>>>>>>>> /* >>>>>>>>> set_param_str_oracle : convert empty string to null value >>>>>>>>> This allow to bind empty string in JDBC as a null value. >>>>>>>>> Ex : >>>>>>>>> String sel="select coalesce(?,'null param') from dual"; >>>>>>>>> PreparedStatement ps_sel = m_cnx.prepareStatement(sel); >>>>>>>>> ps_sel.setString(1, ""); >>>>>>>>> ResultSet rs = ps_sel.executeQuery(); >>>>>>>>> while (rs.next()) >>>>>>>>> { >>>>>>>>> System.out.println(rs.getString(1)); >>>>>>>>> } >>>>>>>>> Returns : 'null param' >>>>>>>>> */ >>>>>>>>> static void set_param_str_oracle(Item_param *param, uchar >> **pos, >>>>>> ulong >>>>>>>>> len) { >>>>>>>>> ulong length= get_param_length(pos, len); >>>>>>>>> if (length == 0) >>>>>>>>> param->set_null(); >>>>>>>>> else >>>>>>>>> { >>>>>>>>> if (length > len) >>>>>>>>> length= len; >>>>>>>>> param->set_str((const char *) *pos, length); >>>>>>>>> *pos+= length; >>>>>>>>> } >>>>>>>>> } >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Jérôme. >>>>>>>>> >>>>>>>>> >>>>>>>>>> -----Message d'origine----- >>>>>>>>>> De : Alexander Barkov [mailto:bar@mariadb.org] Envoyé : mardi >>>>>>>>>> 26 septembre 2017 13:34 À : jerome brauge Objet : Re: Patch for >>>>>>>>>> MDEV-10574 >>>>>>>>>> >>>>>>>>>> Hello Jérôme, >>>>>>>>>> >>>>>>>>>> I'm afraid we cannot accept this change at this point of time. >>>>>>>>>> >>>>>>>>>> The problem is that all around the code we assume that NULL is >>>>>>>>>> not equal to ''. >>>>>>>>>> >>>>>>>>>> Changing '' to NULL only in the visible part of the iceberg >>>>>>>>>> (such as in the parser and in >>>>>>>>>> Item_str_func::make_empty_result()) >>>>>>>>>> will likely introduce various anomalies in fundamental things >>>>>>>>>> like joins execution, optimizer, equality propagation, etc. >>>>>>>>>> >>>>>>>>>> Sorry, we don't have resources to dive into this topic deeper now. >>>>>>>>>> Moreover, Oracle's documentation suggests not to reply on the >>>>>>>>>> fact >>>>>> that '' >>>>>>>>>> and NULL are treated as same, as this can change in the future. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 09/20/2017 03:52 PM, jerome brauge wrote: >>>>>>>>>>> Hello Alexander, >>>>>>>>>>> Here is a new version of this patch (minor optimizations) >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Jérôme. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> -----Message d'origine----- >>>>>>>>>>>> De : Maria-developers [mailto:maria-developers- >>>>>>>>>>>> bounces+j.brauge=qualiac.com(a)lists.launchpad.net] De la part >>>> de >>>>>>>>>>>> bounces+jerome >>>>>>>>>>>> brauge >>>>>>>>>>>> Envoyé : mardi 19 septembre 2017 16:09 À : 'Alexander Barkov' >>>>>>>>>>>> Cc : MariaDB Developers >>>>>>>>>>>> (maria-developers(a)lists.launchpad.net) >>>>>>>>>>>> Objet : [Maria-developers] Patch for MDEV-10574 >>>>>>>>>>>> >>>>>>>>>>>> Alexander, >>>>>>>>>>>> Here is the patch I was talking about. >>>>>>>>>>>> It not address all aspects of MDEV-10574, especially, empty >>>>>>>>>>>> strings in table column are not view as null. >>>>>>>>>>>> But as in oracle mode, no empty string in columns should be >>>>>>>>>>>> created, an fully oracle application will never fall in this case. >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> Jérôme. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> -----Message d'origine----- >>>>>>>>>>>>> De : jerome brauge >>>>>>>>>>>>> Envoyé : mardi 19 septembre 2017 14:13 À : 'Alexander >> Barkov' >>>>>>>>>>>>> Objet : RE: Oracle strings functions compatibility : substr >>>>>>>>>>>>> >>>>>>>>>>>>> Hello Alexander, >>>>>>>>>>>>> Yes I'm aware. >>>>>>>>>>>>> This point will be treated by my next patch which will also >>>>>>>>>>>>> affect the others string functions (a part of MDEV-10574). >>>>>>>>>>>>> I will refresh this new patch with the current development >>>>>>>>>>>>> branch, and I submit to you. >>>>>>>>>>>>> Regards. >>>>>>>>>>>>> Jérôme. >>>>>>>>>>>>> >>>>>>>>>>>>>> -----Message d'origine----- De : Alexander Barkov >>>>>>>>>>>>>> [mailto:bar@mariadb.org] Envoyé : >>>> mardi >>>>>> 19 >>>>>>>>>>>>>> septembre 2017 13:16 À : jerome brauge Objet : Re: Oracle >>>>>>>>>>>>>> strings functions compatibility : substr >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Jerome, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I noticed one more difference in SUBSTR: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> SELECT SUBSTR('aaa',1,-1) FROM DUAL; >>>>>>>>>>>>>> >>>>>>>>>>>>>> MariaDB returns an empty string. >>>>>>>>>>>>>> Oracle returns NULL. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Here's Oracle's documentation: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>> >>>>>> >> https://docs.oracle.com/cd/B19306_01/server.102/b14200/functions162. >>>>>>>>>>>>>> ht >>>>>>>>>>>>>> m >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think we should do this in the same patch, to avoid >>>>>>>>>>>>>> behavior change in the future. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Would you like to try doing this additional change? >>>>>>>>>>>>>> Or would you like me to make an incremental patch on top >> of >>>>>> yours? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks! >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 09/19/2017 10:33 AM, jerome brauge wrote: >>>>>>>>>>>>>>> Hello Alexander, >>>>>>>>>>>>>>> It's done. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Do you have news from Sergei for MDEV-12874, MDEV- >> 13417 >>>>>> and >>>>>>>>>>>> MDEV- >>>>>>>>>>>>>> 13418 ? >>>>>>>>>>>>>> >>>>>>>>>>>>>> I asked Sergei, awaiting for his answer. I'll let you know. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thank you very much. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -----Message d'origine----- De : Alexander Barkov >>>>>>>>>>>>>>>> [mailto:bar@mariadb.org] Envoyé : lundi >>>>>>>>>>>>>>>> 18 septembre 2017 18:12 À : jerome brauge Objet : Re: >>>>>>>>>>>>>>>> Oracle strings functions compatibility : substr >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hello Jerome, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 09/18/2017 05:14 PM, jerome brauge wrote: >>>>>>>>>>>>>>>>> Hello Alexander, >>>>>>>>>>>>>>>>> I don't understand how I've missed such an obvious >>>> solution ! >>>>>>>>>>>>>>>>> Here is the new patch. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks. Now it looks simpler. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Can you please do one small thing: >>>>>>>>>>>>>>>> move get_position() from "public:" to "protected:". >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Ok to make a pull request after this change. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thank you very much! >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Regards. >>>>>>>>>>>>>>>>> Jérôme. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> -----Message d'origine----- De : Alexander Barkov >>>>>>>>>>>>>>>>>> [mailto:bar@mariadb.org] Envoyé : >>>>>> lundi >>>>>>>>>>>>>>>>>> 18 septembre 2017 11:30 À : jerome brauge Objet : Re: >>>>>> Oracle >>>>>>>>>>>>>>>>>> strings functions compatibility : substr >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hello Jerome, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 08/17/2017 04:43 PM, jerome brauge wrote: >>>>>>>>>>>>>>>>>>> Hello Alexander, >>>>>>>>>>>>>>>>>>> Here is a patch for function SUBSTR in >> sql_mode=oracle >>>>>>>>>>>>>>>>>>> (If position is 0, >>>>>>>>>>>>>>>>>> then it is treated as 1). >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I'm thinking of how to get a smaller patch. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Wouldn't it be possible to introduce: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> virtual longlong get_position(); >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> longlong Item_func_substr::get_position() { >>>>>>>>>>>>>>>>>> return args[1]->val_int(); } >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> longlong Item_func_substr_oracle::get_position() >>>>>>>>>>>>>>>>>> { >>>>>>>>>>>>>>>>>> longlong pos= args[1]->val_int(); >>>>>>>>>>>>>>>>>> return pos == 0 ? 1 : pos; } >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> and just replace all calls for args[1]->val_int() to >>>>>> get_position() >>>>>>>> ? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks! >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>>> Jérôme. >>>>>>>>>>>>>>>>>>>

2 2

Re: [Maria-developers] [Commits] e0a1e63: MDEV-9619: Assertion `null_ref_table' failed in virtual table_map Item_direct_view_ref::used_tables() const on 2nd execution of PS
by Sergei Golubchik 13 Oct '17

13 Oct '17

Hi, Oleksandr! On Feb 18, Oleksandr Byelkin wrote: > revision-id: e0a1e632e4e6084cd989f217943618f4e9ac9179 (mariadb-5.5.54-25-ge0a1e63) > parent(s): 29d78dbb44ee9890b6bc28873344f20fc9157928 > committer: Oleksandr Byelkin > timestamp: 2017-02-18 17:47:31 +0100 > message: > > MDEV-9619: Assertion `null_ref_table' failed in virtual table_map > Item_direct_view_ref::used_tables() const on 2nd execution of PS > > Refer left expression indirectly in case it changes from execution to > execution. Looks reasonable. Ok to push with these changes: 1. replace - DBUG_ASSERT(null_ref_table); + DBUG_ASSERT(fixed); as discussed. 2. Remove bool Item_direct_view_ref::fix_fields(THD *thd, Item **reference) { - DBUG_ASSERT(1); these two changes should, probably, be done in a separate cleanup commit. 3. > diff --git a/sql/item.h b/sql/item.h > index 89155ac..dff7bc4 100644 > --- a/sql/item.h > +++ b/sql/item.h > @@ -1543,6 +1543,8 @@ class Item { > virtual void mark_as_condition_AND_part(TABLE_LIST *embedding) {}; > }; > > +typedef Item (*Item_ptr); > + remove this, your patch compiles just fine without it. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] Please review MDEV-10802 TIMESTAMP NOT NULL field with explicit_defaults_for_timestamp and NO_ZERO_DATE shouldn't throw error
by Alexander Barkov 13 Oct '17

13 Oct '17

Hi Sergei, Please review a patch for MDEV-10802. Thanks!

2 2

Re: [Maria-developers] [Commits] 06bb7ee: MDEV-10892 - rpl.rpl_semi_sync_uninstall_plugin fails with Assertion `0' failure
by Sergei Golubchik 13 Oct '17

13 Oct '17

Hi, Sergey! On Oct 12, Sergey Vojtovich wrote: > revision-id: 06bb7eef9882536fc1d99b4462a65b53ec9f137e (mariadb-5.5.57-14-g06bb7ee) > parent(s): 93aadda513d8b5c2b49001514e235c4fdd73e08a > committer: Sergey Vojtovich > timestamp: 2017-10-12 15:37:45 +0400 > message: > > MDEV-10892 - rpl.rpl_semi_sync_uninstall_plugin fails with Assertion `0' failure > in buildbot > > Removed plugin_array_version: it is being checked without mutex protection and > thus is prone to data race and race conditions. In effect plugins are not > protected from concurrent destruction. > > Removed state_mask inversion: doesn't seem to make any sense. > > When collecting local plugins list, only add plugins that match state_mask. > > Use plugin ref counting to protect against concurrent plugin destruction. > diff --git a/sql/sql_show.cc b/sql/sql_show.cc > index 6e04564..512548b 100644 > --- a/sql/sql_show.cc > +++ b/sql/sql_show.cc > @@ -262,7 +262,7 @@ int fill_plugins(THD *thd, TABLE_LIST *tables, COND *cond) > TABLE *table= tables->table; > > if (plugin_foreach_with_mask(thd, show_plugins, MYSQL_ANY_PLUGIN, > - ~PLUGIN_IS_FREED, table)) > + ~(PLUGIN_IS_FREED | PLUGIN_IS_DYING), table)) > DBUG_RETURN(1); > > DBUG_RETURN(0); > @@ -5294,7 +5294,8 @@ int fill_schema_engines(THD *thd, TABLE_LIST *tables, COND *cond) > DBUG_ENTER("fill_schema_engines"); > if (plugin_foreach_with_mask(thd, iter_schema_engines, > MYSQL_STORAGE_ENGINE_PLUGIN, > - ~PLUGIN_IS_FREED, tables->table)) > + ~(PLUGIN_IS_FREED | PLUGIN_IS_DYING), > + tables->table)) > DBUG_RETURN(1); > DBUG_RETURN(0); > } agree, ok to push Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] 96e982b: MDEV-7140 - Sporadic buildbot failures in rpl.rpl_semi_sync_uninstall_plugin
by Sergei Golubchik 13 Oct '17

13 Oct '17

Hi, Sergey! On Oct 12, Sergey Vojtovich wrote: > revision-id: 96e982bb1a72db47a41b97bb632b595b347d2956 (mariadb-5.5.57-14-g96e982b) > parent(s): 93aadda513d8b5c2b49001514e235c4fdd73e08a > committer: Sergey Vojtovich > timestamp: 2017-10-12 14:42:17 +0400 > message: > > MDEV-7140 - Sporadic buildbot failures in rpl.rpl_semi_sync_uninstall_plugin > > Removed plugin_array_version: it is being checked without mutex protection and > thus is prone to data race and race conditions. In effect plugins are not > protected from concurrent destruction. > > Removed state_mask inversion: doesn't seem to make any sense. > > When collecting local plugins list, only add plugins that match state_mask. > > Use plugin ref counting to protect against concurrent plugin destruction. ok to push Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MDEV-13530 VARBINARY doesn't convert to to BLOB for sizes 65533, 65534 and 65535
by Alexander Barkov 12 Oct '17

12 Oct '17

Hi Sergei, Please review a patch for MDEV-13530. Thanks!

1 0

Re: [Maria-developers] 46b93fc: MDEV-13676: Field "create Procedure" is NULL, even if the the user has role which is the definer. (SHOW CREATE PROCEDURE)
by Sergei Golubchik 11 Oct '17

11 Oct '17

Hi, Vicentiu! Looks good! A couple of comments, see below. Ok to push after fixing, the second review is not needed. On Oct 09, Vicentiu Ciorbaru wrote: > revision-id: 46b93fc2a2fa198d721813d9b93bbe09d2e36eb3 (mariadb-10.0.32-50-g46b93fc) > parent(s): dbeffabc83ed01112e09d7e782d44f044cfcb691 > author: Vicențiu Ciorbaru > committer: Vicențiu Ciorbaru > timestamp: 2017-10-09 13:32:40 +0300 > message: > > MDEV-13676: Field "create Procedure" is NULL, even if the the user has role which is the definer. (SHOW CREATE PROCEDURE) > > During show create procedure we ommited to check the current role, if it > is the actual definer of the procedure. In addition, we should support > indirectly granted roles to the current role. Implemented a recursive > lookup to search the tree of grants if the rolename is present. We both had to check the SQL standard to see what the behavior should be. Please add here (in the commit comment) a reference to the exact part of the standard that specifies this behavior. Like SQL Standard 2016, Part ... Section ... View I_S.ROUTINES selects ROUTINE_BODY and its WHERE clause says that the GRANTEE must be either PUBLIC, or CURRENT_USER or in the ENABLED_ROLES. > diff --git a/sql/sp_head.cc b/sql/sp_head.cc > index ea9e1c1..3dd1a65 100644 > --- a/sql/sp_head.cc > +++ b/sql/sp_head.cc > @@ -2588,10 +2588,18 @@ bool check_show_routine_access(THD *thd, sp_head *sp, bool *full_access) > *full_access= ((!check_table_access(thd, SELECT_ACL, &tables, FALSE, > 1, TRUE) && > (tables.grant.privilege & SELECT_ACL) != 0) || > + /* Check if user owns the routine. */ > (!strcmp(sp->m_definer_user.str, > thd->security_ctx->priv_user) && > !strcmp(sp->m_definer_host.str, > - thd->security_ctx->priv_host))); > + thd->security_ctx->priv_host)) || > + /* Check if current role or any of the sub-granted roles > + own the routine. */ > + (sp->m_definer_host.length == 0 && > + (!strcmp(sp->m_definer_user.str, > + thd->security_ctx->priv_role) || > + check_role_is_granted(thd->security_ctx->priv_role, NULL, > + sp->m_definer_user.str)))); Perhaps you could simplify the condition above a little bit, by replacing it with /* Check if current role or any of the sub-granted roles own the routine. */ (sp->m_definer_host.length == 0 && check_role_is_granted(thd->security_ctx->priv_user, thd->security_ctx->priv_host, sp->m_definer_user.str))); Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 1

[Maria-developers] MDEV-14038 ALTER TABLE does not exit on error with InnoDB + bad default function
by Alexander Barkov 11 Oct '17

11 Oct '17

Hello Marko, please review a patch for MDEV-14038. Thanks!

2 1

[Maria-developers] MDEV-13972 crash in Item_func_sec_to_time::get_date
by Alexander Barkov 09 Oct '17

09 Oct '17

Hello Sergei, Please review a patch for MDEV-13972. The patch is for 5.5, as we agreed in slack. Thanks!

2 3

Re: [Maria-developers] [Commits] 0240b866abc: MDEV-13838: Wrong result after altering a partitioned table
by Marko Mäkelä 09 Oct '17

09 Oct '17

On Mon, Oct 9, 2017 at 3:00 PM, jan <jan.lindstrom(a)mariadb.com> wrote: > revision-id: 109463fdb86d (mariadb-10.0.32-54-g0240b866abc) > parent(s): 172cc70bf8c0aea3d2d0c73bcf94f36c172b769a > author: Jan Lindström > committer: Jan Lindström > timestamp: 2017-10-09 14:52:51 +0300 > message: > > MDEV-13838: Wrong result after altering a partitioned table > > Reverted incorrect changes done on MDEV-7367 and MDEV-9469. Fixes properly > also related bugs: > > MDEV-13668: InnoDB unnecessarily rebuilds table when renaming a column and > adding index > MDEV-9469: 'Incorrect key file' on ALTER TABLE > MDEV-9548: Alter table (renaming and adding index) fails with "Incorrect > key file for table" > MDEV-10535: ALTER TABLE causes standalone/wsrep cluster crash > MDEV-13640: ALTER TABLE CHANGE and ADD INDEX on auto_increment column > fails with "Incorrect key file for table..." > This is a very welcome change that looks correct to me. I only have a few comments on the formatting and tests. The patch does not cleanly apply to 10.0 due to some difference in the comments of removed functions. Later, you informed me that this is https://github.com/MariaDB/server/commit/a8db085c51ccfd6df737d9261abac9d2b0… on bb-10.0-MDEV-13838 <https://github.com/MariaDB/server/compare/bb-10.0-MDEV-13838>. +Warnings: > +Note 1831 Duplicate index `c4_2`. This is deprecated and will be > disallowed in a future release. > The intention of the test was not to create duplicated indexes. The "rename column" and "add index" operations were split into two ALTER TABLE in order to work around the limitation that was introduced in a previous incorrect commit. You should have removed the added ALTER TABLE statements instead of adding SHOW CREATE TABLE statements. Please refer to https://github.com/MariaDB/server/commit/f56bd70f51020c67a30415ae381ae93b90… and revert the relevant parts of the test changes. The other tests seem to be adjusted fine. > @@ -171,3 +172,39 @@ ALTER TABLE ticket > SHOW CREATE TABLE ticket; > > DROP TABLE ticket; > + > +# > +# MDEV-13838: Wrong result after altering a partitioned table > +# > + > +CREATE TABLE `t` ( > +`id` bigint(20) unsigned NOT NULL auto_increment, > +`d` date NOT NULL, > +`a` bigint(20) unsigned NOT NULL, > +`b` smallint(5) unsigned DEFAULT NULL, > +PRIMARY KEY (`id`,`d`) > +) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_general_cs > STATS_SAMPLE_PAGES=200 > +/*!50500 PARTITION BY RANGE COLUMNS(d) > +(PARTITION p20170913 VALUES LESS THAN ('2017-09-14') ENGINE = InnoDB, > +PARTITION p20170914 VALUES LESS THAN ('2017-09-15') ENGINE = InnoDB, > +PARTITION p20170915 VALUES LESS THAN ('2017-09-16') ENGINE = InnoDB, > +PARTITION p20170916 VALUES LESS THAN ('2017-09-17') ENGINE = InnoDB, > +PARTITION p20170917 VALUES LESS THAN ('2017-09-18') ENGINE = InnoDB, > +PARTITION p99991231 VALUES LESS THAN (MAXVALUE) ENGINE = InnoDB) */; > Please clean up the CREATE TABLE statement. Remove trailing space and unnecessary quotes and comments. > +insert into t(d,a,b) values ('2017-09-15',rand()*10000,rand()*10); > +insert into t(d,a,b) values ('2017-09-15',rand()*10000,rand()*10); > + > +replace into t(d,a,b) select '2017-09-15',rand()*10000,rand()*10 from t > t1, t t2, t t3, t t4, t t5, t t6, t t7, t t8, t t9, t t10, t t11, t t12, t > t13, t t14; > Does the table really have to be this big for the bug to repeat? Can you please reduce the test? Shouldn’t it work with 1 or 2 partitions and a few records in each? > + > +select count(*) from t where d ='2017-09-15'; > + > +ALTER TABLE t CHANGE b c smallint(5) unsigned , ADD KEY idx_d_a (d, a); > +SHOW CREATE TABLE t; > +analyze table t; > + > +select count(*) from t where d ='2017-09-15'; > +select count(*) from t force index(primary) where d ='2017-09-15'; > + > +DROP TABLE t; > + > Please do not end files in empty lines. > @@ -1625,13 +1606,9 @@ innobase_create_index_def( > index->ind_type |= DICT_FTS; > } > > - if (!new_clustered) { > - altered_table = NULL; > - } > - > for (i = 0; i < n_fields; i++) { > - innobase_create_index_field_def( > - altered_table, &key->key_part[i], > &index->fields[i], fields); > + innobase_create_index_field_def(new_clustered, > + altered_table, &key->key_part[i], > &index->fields[i]); > } > > DBUG_VOID_RETURN; > Do not add the first parameter to the same line with the function name. Instead, add it on a line on its own. In this way, it follows the indentation rules set by (defun innodb-cc-mode (&optional map) "InnoDB adjustments to cc-mode. In MAP, binds C-c C-c to 'compile." (if (and (buffer-file-name) (string-match "/\$innobase\\|xtradb\$.*/.*[ihc]\\'" (buffer-file-name))) (progn (setq mode-name (concat "Ib" mode-name) c-basic-offset 8 c-label-minimum-indentation 0) (add-to-list 'c-offsets-alist '(c . 0)) (add-to-list 'c-offsets-alist '(arglist-intro . +)) (add-to-list 'c-offsets-alist '(label . [0])))) (if map (define-key map "\C-c\C-c" 'compile))) > @@ -2121,7 +2097,7 @@ struct ha_innobase_inplace_ctx : public > inplace_alter_handler_ctx > /** mapping of old column numbers to new ones, or NULL */ > const ulint* col_map; > /** new column names, or NULL if nothing was renamed */ > - const char** col_names; > + const char** col_names; > /** added AUTO_INCREMENT column position, or ULINT_UNDEFINED */ > const ulint add_autoinc; > /** default values of ADD COLUMN, or NULL */ > Please do not change TAB to spaces. > @@ -3067,7 +3043,7 @@ prepare_inplace_alter_table_dict( > > ctx->add_index[a] = row_merge_create_index( > ctx->trx, ctx->new_table, > - &index_defs[a], ctx->col_names); > + &index_defs[a]); > > add_key_nums[a] = index_defs[a].key_number; > Consider joining the line after new_table. Best regards, Marko

1 0

[Maria-developers] MDEV-11819 NO_ZERO_IN_DATE: Incorrect generated column value
by Alexander Barkov 06 Oct '17

06 Oct '17

Hello Sergei, Please review a patch for MDEV-11819. Thanks.

2 1

[Maria-developers] Add RMariaDB CRAN Link to MariaDB Foundation Download Webpage
by Juan Telleria 06 Oct '17

06 Oct '17

Dear R Developers, A new package has been released for the "R Statistical Programming Language" in which the R community has put a great effort into it: RMariaDB. It enables a robust data transfer between R and MariaDB. The link on CRAN (R package repository) can be accessed from: https://cran.r-project.org/web/packages/RMariaDB/index.html And can be installed in R by using the following statement: install.packages("RMariaDB") library(RMariaDB) So, just as PHP community developed connector was distributed in the past within MySQL webpage, I do think that this connector could benefit from being distributed (or having a link to CRAN) from the MariaDB Foundation Download Webpage. I believe that many people has putted a great effort on this connector, and it would benefit from greater visibility for better support and maintenance. Thank you, Juan

2 1

Re: [Maria-developers] [Commits] 3f7e619647f: MDEV-13928: Missing symbols building RocksDB on macOS 10.12.6
by Sergei Golubchik 05 Oct '17

05 Oct '17

Hi, Varun! Why did it fail only on MacOS X? On Oct 06, Varun wrote: > revision-id: 3f7e619647f7bc1ec31a6118b403da805c48ae51 (mariadb-10.2.5-653-g3f7e619647f) > parent(s): 08c493c62afae0e81f9da6bb9b86efe99a9a763d > author: Varun Gupta > committer: Varun Gupta > timestamp: 2017-10-06 00:20:08 +0530 > message: > > MDEV-13928: Missing symbols building RocksDB on macOS 10.12.6 > > rdb_sst_info.cc must be compiled with RTTI > > --- > storage/rocksdb/CMakeLists.txt | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/storage/rocksdb/CMakeLists.txt b/storage/rocksdb/CMakeLists.txt > index 6db82119b45..80a6614b757 100644 > --- a/storage/rocksdb/CMakeLists.txt > +++ b/storage/rocksdb/CMakeLists.txt > @@ -170,6 +170,7 @@ IF(CMAKE_CXX_COMPILER_ID MATCHES "GNU" OR CMAKE_CXX_COMPILER_ID MATCHES "Clang") > # includes requires it. So, now everything in MariaRocks is compiled with > # -frtti) > set_source_files_properties(event_listener.cc rdb_cf_options.cc > + rdb_sst_info.cc > PROPERTIES COMPILE_FLAGS -frtti) > ENDIF() Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MDEV-13417 / MDEV - 13418 The order of evaluation of SELECT..INTO assignments
by jerome brauge 03 Oct '17

03 Oct '17

Hello Sergei, Here is a patch for MDEV-13417 and MDEV-13418. Until you choose the right name for this new sql_mode, I temporarily chosen "SET_CONSISTENCY". It is set only for sql_mode=oracle for now. Can you review it ? Thank you. Regard.

2 6

[Maria-developers] MDEV-10596
by jerome brauge 27 Sep '17

27 Sep '17

Hi Alexander, In sql_mode=oracle, when a stored procedure varchar parameter is defined without size, you have chosen to translate it by VARCHAR(4000) (or CHAR(2000) for CHAR parameter). Oracle does not work like this. Size is inherited from the size of argument at runtime. Example: CREATE OR REPLACE PROCEDURE p1(p OUT VARCHAR) AS BEGIN p:='0123456789'; END; / declare w varchar(10); begin p1(w); end; / --> work fine declare w varchar(8); begin p1(w); end; / --> failed with : ERROR at line 1: ORA-06502: PL/SQL: numeric or value error: character string buffer too small ORA-06512: at "OPS$DBO.P1", line 4 ORA-06512: at line 3 Furthermore, since Oracle 9, VARCHAR datatype in PL/SQL is not limited to 4000 char but to 32k. It's the size of varchar column in a table that is limited to 4000 (until Oracle 12C which allow 32k when MAX_STRING_SIZE=EXTENDED). I've done the attached patch to resize these parameters at runtime. What do you think about it ? Another question : Oracle does not allow to change the value of an IN parameter. The following procedure cannot be compile : CREATE OR REPLACE PROCEDURE p1(p IN VARCHAR) AS BEGIN p:='0123456789'; END; / Warning: Procedure created with compilation errors. //sun10:1521/CS> select * from user_errors; NAME TYPE SEQUENCE LINE POSITION ------------------------------ ------------ ---------- ---------- ---------- TEXT ------------------------------------------------------------------------------------------------------------------------------------ ATTRIBUTE MESSAGE_NUMBER --------- -------------- P1 PROCEDURE 1 4 3 PLS-00363: expression 'P' cannot be used as an assignment target ERROR 363 Mariadb allow this. It's fine for us but do you have plan to change this behavior ? (or add a new STRICT_xxx mode ?) Regard, Jérôme.

2 3

[Maria-developers] GCC7 warnings
by Reindl Harald 24 Sep '17

24 Sep '17

hard to say how serious they are but maybe a good idea to take a look /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/libmariadb/mariadb_config/mariadb_config.c:9:49: warning: backslash and newline separated by space #define LIBS "-L/usr/lib64/mysql/ -lmariadb "\ /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/libmariadb/libmariadb/mariadb_lib.c: In function 'mariadb_get_infov': /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/libmariadb/libmariadb/mariadb_lib.c:3872:30: warning: this statement may fall through [-Wimplicit-fallthrough=] *((unsigned long *)arg)= mysql->client_flag; ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~ /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/libmariadb/libmariadb/mariadb_lib.c:3875:3: note: here default: ^~~~~~~ In file included from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/plugin/handler_socket/libhsclient/hstcpcli.cpp:13:0: /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/plugin/handler_socket/libhsclient/hstcpcli.hpp:33:14: warning: 'template<class> class std::auto_ptr' is deprecated [-Wdeprecated-declarations] typedef std::auto_ptr<hstcpcli_i> hstcpcli_ptr; ^~~~~~~~ In file included from /usr/include/c++/7/memory:80:0, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/plugin/handler_socket/libhsclient/hstcpcli.hpp:15, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/plugin/handler_socket/libhsclient/hstcpcli.cpp:13: /usr/include/c++/7/bits/unique_ptr.h:51:28: note: declared here template<typename> class auto_ptr; /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/libmariadb/libmariadb/ma_dtoa.c: In function 'dtoa': /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/libmariadb/libmariadb/ma_dtoa.c:1465:14: warning: this statement may fall through [-Wimplicit-fallthrough=] leftright= 0; ~~~~~~~~~^~~ /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/libmariadb/libmariadb/ma_dtoa.c:1467:3: note: here case 4: ^~~~ /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/libmariadb/libmariadb/ma_dtoa.c:1473:14: warning: this statement may fall through [-Wimplicit-fallthrough=] leftright= 0; ~~~~~~~~~^~~ /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/libmariadb/libmariadb/ma_dtoa.c:1475:3: note: here case 5: ^~~~ At top level: /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/libmariadb/libmariadb/ma_dtoa.c:1161:21: warning: 'tinytens' defined but not used [-Wunused-const-variable=] static const double tinytens[]= ^~~~~~~~ /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/libmariadb/libmariadb/mariadb_dyncol.c: In function 'mariadb_dyncol_val_double': /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/libmariadb/libmariadb/mariadb_dyncol.c:4044:7: warning: this statement may fall through [-Wimplicit-fallthrough=] { ^ /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/libmariadb/libmariadb/mariadb_dyncol.c:4060:5: note: here case DYN_COL_DATETIME: ^~~~ /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/libmariadb/libmariadb/mariadb_dyncol.c:4050:15: warning: argument 1 null where non-null expected [-Wnonnull] *dbl= strtod(str, &end); ^~~~~~~~~~~~~~~~~ In file included from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/libmariadb/include/ma_global.h:168:0, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/libmariadb/libmariadb/mariadb_dyncol.c:64: /usr/include/stdlib.h:126:15: note: in a call to function 'strtod' declared here extern double strtod (const char *__restrict __nptr, ^~~~~~ /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/libmariadb/unittest/libmariadb/t_aurora.c: In function 'test_reconnect': /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/libmariadb/unittest/libmariadb/t_aurora.c:100:3: warning: ignoring return value of 'system', declared with attribute warn_unused_result [-Wunused-result] system("/usr/local/aws/bin/aws rds failover-db-cluster --db-cluster-identifier instance-1-cluster"); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/mysys/my_lib.c: In function 'my_dir': /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/mysys/my_lib.c:144:3: warning: 'readdir_r' is deprecated [-Wdeprecated-declarations] while (!(READDIR(dirp,(struct dirent*) dirent_tmp,dp))) ^~~~~ In file included from /usr/include/features.h:410:0, from /usr/include/pthread.h:21, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/include/my_global.h:278, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/mysys/mysys_priv.h:19, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/mysys/my_lib.c:19: /usr/include/dirent.h:189:12: note: declared here extern int __REDIRECT (readdir_r, /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/storage/myisam/myisamchk.c: In function 'myisamchk': /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/storage/myisam/myisamchk.c:1411:22: warning: '%ld' directive writing between 1 and 10 bytes into a region of size 8 [-Wformat-overflow=] sprintf(null_pos,"%ld",(long) keyseg->null_pos+1); ^~~ /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/storage/myisam/myisamchk.c:1411:21: note: directive argument in the range [1, 4294967296] sprintf(null_pos,"%ld",(long) keyseg->null_pos+1); ^~~~~ In file included from /usr/include/stdio.h:939:0, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/include/my_global.h:322, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/include/my_base.h:23, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/include/myisam.h:26, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/storage/myisam/myisamdef.h:20, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/storage/myisam/fulltext.h:21, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/storage/myisam/myisamchk.c:18: /usr/include/bits/stdio2.h:33:10: note: '__builtin___sprintf_chk' output between 2 and 11 bytes into a destination of size 8 return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ __bos (__s), __fmt, __va_arg_pack ()); ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/storage/myisam/mi_test3.c: In function 'test_write': /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/storage/myisam/mi_test3.c:367:30: warning: '%7ld' directive writing between 7 and 11 bytes into a region of size 8 [-Wformat-overflow=] sprintf((char*) record.id,"%7ld",(long) getpid()); ^~~~ /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/storage/myisam/mi_test3.c:367:29: note: directive argument in the range [-2147483648, 2147483647] sprintf((char*) record.id,"%7ld",(long) getpid()); ^~~~~~ In file included from /usr/include/stdio.h:939:0, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/include/my_global.h:322, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/include/my_base.h:23, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/include/myisam.h:26, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/storage/myisam/mi_test3.c:20: /usr/include/bits/stdio2.h:33:10: note: '__builtin___sprintf_chk' output between 8 and 12 bytes into a destination of size 8 return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ __bos (__s), __fmt, __va_arg_pack ()); /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/storage/maria/maria_chk.c: In function 'maria_chk': /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/storage/maria/maria_chk.c:1703:22: warning: '%ld' directive writing between 1 and 10 bytes into a region of size 8 [-Wformat-overflow=] sprintf(null_pos,"%ld",(long) keyseg->null_pos+1); ^~~ /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/storage/maria/maria_chk.c:1703:21: note: directive argument in the range [1, 4294967296] sprintf(null_pos,"%ld",(long) keyseg->null_pos+1); ^~~~~ In file included from /usr/include/stdio.h:939:0, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/include/my_global.h:322, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/include/my_base.h:23, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/include/maria.h:22, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/storage/maria/maria_def.h:18, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/storage/maria/ma_fulltext.h:20, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/storage/maria/maria_chk.c:18: /usr/include/bits/stdio2.h:33:10: note: '__builtin___sprintf_chk' output between 2 and 11 bytes into a destination of size 8 return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ __bos (__s), __fmt, __va_arg_pack ()); ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/storage/maria/ma_test3.c: In function 'test_write': /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/storage/maria/ma_test3.c:365:30: warning: '%7ld' directive writing between 7 and 11 bytes into a region of size 8 [-Wformat-overflow=] sprintf((char*) record.id,"%7ld", (long) getpid()); ^~~~ /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/storage/maria/ma_test3.c:365:29: note: directive argument in the range [-2147483648, 2147483647] sprintf((char*) record.id,"%7ld", (long) getpid()); ^~~~~~ In file included from /usr/include/stdio.h:939:0, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/include/my_global.h:322, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/include/my_base.h:23, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/include/maria.h:22, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/storage/maria/maria_def.h:18, from /home/builduser/rpmbuild/BUILD/mariadb-10.2.8/storage/maria/ma_test3.c:20: /usr/include/bits/stdio2.h:33:10: note: '__builtin___sprintf_chk' output between 8 and 12 bytes into a destination of size 8 return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ __bos (__s), __fmt, __va_arg_pack ()); ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

1 0

Re: [Maria-developers] [Commits] 2846574452e: MDEV-11553 Can't restore a PERSISTENT column that uses DATE_FORMAT()
by Alexander Barkov 23 Sep '17

23 Sep '17

Hello Sergei, Ok to push. One small suggestion. Instead of having a standalone function, please move get_locale as method in Item. Something like this: const MY_LOCALE *Item::get_locale() { StringBuffer<MAX_FIELD_WIDTH> tmp; String *locale_name= val_str_ascii(&tmp); const MY_LOCALE *lc; if (!locale_name || !(lc= my_locale_by_name(locale_name->c_ptr_safe()))) { THD *thd= current_thd; push_warning_printf(thd, Sql_condition::WARN_LEVEL_WARN, ER_UNKNOWN_LOCALE, ER_THD(thd, ER_UNKNOWN_LOCALE), locale_name ? locale_name->c_ptr_safe() : "NULL"); lc= &my_locale_en_US; } return lc; } Thanks! On 09/22/2017 10:30 PM, serg(a)mariadb.org wrote: > revision-id: 2846574452e82b9f391ca5d41d5021ac94730dec (base-0.6-627-g2846574452e) > parent(s): 5a8b6e38967f1e4aa7fe4885e54ee8ea44097e8e > author: Sergei Golubchik > committer: Sergei Golubchik > timestamp: 2017-09-22 20:24:53 +0200 > message: > > MDEV-11553 Can't restore a PERSISTENT column that uses DATE_FORMAT() > > 3-argument form of DATE_FORMAT > > --- > mysql-test/r/locale.result | 71 +++++++++++++++++++++- > .../suite/vcol/r/vcol_blocked_sql_funcs.result | 2 + > .../suite/vcol/r/vcol_supported_sql_funcs.result | 15 +++++ > .../suite/vcol/t/vcol_blocked_sql_funcs_main.inc | 3 + > .../suite/vcol/t/vcol_supported_sql_funcs_main.inc | 6 ++ > mysql-test/t/locale.test | 28 +++++++++ > sql/item_create.cc | 27 +------- > sql/item_strfunc.cc | 10 +-- > sql/item_strfunc.h | 4 +- > sql/item_timefunc.cc | 25 ++++++-- > sql/item_timefunc.h | 27 +++++--- > sql/lex.h | 1 + > sql/sql_yacc.yy | 16 ++++- > sql/sql_yacc_ora.yy | 16 ++++- > 14 files changed, 202 insertions(+), 49 deletions(-) > > diff --git a/mysql-test/r/locale.result b/mysql-test/r/locale.result > index a02e80ed21e..b750d038205 100644 > --- a/mysql-test/r/locale.result > +++ b/mysql-test/r/locale.result > @@ -229,15 +229,80 @@ SET lc_time_names=@old_50915_lc_time_names; > # End of 10.1 tests > # > create view v1 as select > +date_format('2001-10-02', '%c %b %M') as a, > +date_format('2001-10-02', '%c %b %M', 'ru_RU') as a1, > format(123456789,2) as b, > format(123456789,2,'rm_CH') as b1; > select * from v1; > -b b1 > -123,456,789.00 123'456'789,00 > +a a1 b b1 > +10 Oct October 10 Окт Октября 123,456,789.00 123'456'789,00 > show create view v1; > View Create View character_set_client collation_connection > -v1 CREATE ALGORITHM=UNDEFINED DEFINER=`root`@`localhost` SQL SECURITY DEFINER VIEW `v1` AS select format(123456789,2) AS `b`,format(123456789,2,'rm_CH') AS `b1` utf8 utf8_general_ci > +v1 CREATE ALGORITHM=UNDEFINED DEFINER=`root`@`localhost` SQL SECURITY DEFINER VIEW `v1` AS select date_format('2001-10-02','%c %b %M') AS `a`,date_format('2001-10-02','%c %b %M','ru_RU') AS `a1`,format(123456789,2) AS `b`,format(123456789,2,'rm_CH') AS `b1` utf8 utf8_general_ci > drop view v1; > # > # End of 10.2 tests > # > +select date_format('2001-01-01', '%w %a %W', 'ro_RO'); > +date_format('2001-01-01', '%w %a %W', 'ro_RO') > +1 Lu Luni > +select date_format('2001-01-03', '%w %a %W', 'ro_RO'); > +date_format('2001-01-03', '%w %a %W', 'ro_RO') > +3 Mi Miercuri > +select date_format('2001-01-05', '%w %a %W', 'ro_RO'); > +date_format('2001-01-05', '%w %a %W', 'ro_RO') > +5 Vi Vineri > +select date_format('2001-01-07', '%w %a %W', 'ro_RO'); > +date_format('2001-01-07', '%w %a %W', 'ro_RO') > +0 Du Duminică > +select date_format('2001-01-01', '%w %a %W', 'de_AT'); > +date_format('2001-01-01', '%w %a %W', 'de_AT') > +1 Mon Montag > +select date_format('2001-02-01', '%w %a %W', 'de_AT'); > +date_format('2001-02-01', '%w %a %W', 'de_AT') > +4 Don Donnerstag > +select date_format('2001-03-01', '%w %a %W', 'de_AT'); > +date_format('2001-03-01', '%w %a %W', 'de_AT') > +4 Don Donnerstag > +select date_format('2001-01-01', '%w %a %W', 'en_US'); > +date_format('2001-01-01', '%w %a %W', 'en_US') > +1 Mon Monday > +select date_format('2001-03-01', '%c %b %M', 'en_US'); > +date_format('2001-03-01', '%c %b %M', 'en_US') > +3 Mar March > +select date_format('2001-01-01', '%w %a %W', 'rm_CH'); > +date_format('2001-01-01', '%w %a %W', 'rm_CH') > +1 gli glindesdi > +select date_format('2001-01-03', '%w %a %W', 'rm_CH'); > +date_format('2001-01-03', '%w %a %W', 'rm_CH') > +3 me mesemna > +select date_format('2001-01-05', '%w %a %W', 'rm_CH'); > +date_format('2001-01-05', '%w %a %W', 'rm_CH') > +5 ve venderdi > +select date_format('2001-01-07', '%w %a %W', 'rm_CH'); > +date_format('2001-01-07', '%w %a %W', 'rm_CH') > +0 du dumengia > +select date_format('2001-02-01', '%c %b %M', 'rm_CH'); > +date_format('2001-02-01', '%c %b %M', 'rm_CH') > +2 favr favrer > +select date_format('2001-04-01', '%c %b %M', 'rm_CH'); > +date_format('2001-04-01', '%c %b %M', 'rm_CH') > +4 avr avrigl > +select date_format('2001-06-01', '%c %b %M', 'rm_CH'); > +date_format('2001-06-01', '%c %b %M', 'rm_CH') > +6 zercl zercladur > +select date_format('2001-08-01', '%c %b %M', 'rm_CH'); > +date_format('2001-08-01', '%c %b %M', 'rm_CH') > +8 avust avust > +select date_format('2001-10-01', '%c %b %M', 'rm_CH'); > +date_format('2001-10-01', '%c %b %M', 'rm_CH') > +10 oct october > +select date_format('2001-12-01', '%c %b %M', 'rm_CH'); > +date_format('2001-12-01', '%c %b %M', 'rm_CH') > +12 dec december > +select date_format('2001-01-06', '%w %a %W', 'de_CH'); > +date_format('2001-01-06', '%w %a %W', 'de_CH') > +6 Sa Samstag > +select date_format('2001-09-01', '%c %b %M', 'de_CH'); > +date_format('2001-09-01', '%c %b %M', 'de_CH') > +9 Sep September > diff --git a/mysql-test/suite/vcol/r/vcol_blocked_sql_funcs.result b/mysql-test/suite/vcol/r/vcol_blocked_sql_funcs.result > index 6605f7c99b4..2d5bcfe553d 100644 > --- a/mysql-test/suite/vcol/r/vcol_blocked_sql_funcs.result > +++ b/mysql-test/suite/vcol/r/vcol_blocked_sql_funcs.result > @@ -247,3 +247,5 @@ ERROR HY000: Expression in the GENERATED ALWAYS AS clause is too big > # Constant expression > create or replace table t1 (a int as (PI()) PERSISTENT); > drop table if exists t1; > +create table t1 (a timestamp, b varchar(255) as (date_format(a, '%w %a %m %b')) stored); > +ERROR HY000: Function or expression 'date_format()' cannot be used in the GENERATED ALWAYS AS clause of `b` > diff --git a/mysql-test/suite/vcol/r/vcol_supported_sql_funcs.result b/mysql-test/suite/vcol/r/vcol_supported_sql_funcs.result > index 3fa4f6e1431..10ab32f9486 100644 > --- a/mysql-test/suite/vcol/r/vcol_supported_sql_funcs.result > +++ b/mysql-test/suite/vcol/r/vcol_supported_sql_funcs.result > @@ -2894,6 +2894,21 @@ a b > 2012-12-12 Wednesday Wed December Dec > drop table t1; > set sql_warnings = 0; > +# DATE_FORMAT() STORED > +set sql_warnings = 1; > +create table t1 (a date, b varchar(100) as (date_format(a, '%W %a %M %b', 'de_DE')) STORED); > +show create table t1; > +Table Create Table > +t1 CREATE TABLE `t1` ( > + `a` date DEFAULT NULL, > + `b` varchar(100) GENERATED ALWAYS AS (date_format(`a`,'%W %a %M %b','de_DE')) STORED > +) ENGINE=MyISAM DEFAULT CHARSET=latin1 > +insert into t1 values ('2012-12-12',default); > +select * from t1; > +a b > +2012-12-12 Mittwoch Mi Dezember Dez > +drop table t1; > +set sql_warnings = 0; > # CURRENT_USER() > set sql_warnings = 1; > create table t1 (a char, b varchar(32) as (current_user())); > diff --git a/mysql-test/suite/vcol/t/vcol_blocked_sql_funcs_main.inc b/mysql-test/suite/vcol/t/vcol_blocked_sql_funcs_main.inc > index 766d0c7410c..37de02a2b29 100644 > --- a/mysql-test/suite/vcol/t/vcol_blocked_sql_funcs_main.inc > +++ b/mysql-test/suite/vcol/t/vcol_blocked_sql_funcs_main.inc > @@ -354,3 +354,6 @@ eval create or replace table t1 (a int, b varchar(16384) as (concat(a,'$tmp_long > create or replace table t1 (a int as (PI()) PERSISTENT); > > drop table if exists t1; > + > +--error ER_GENERATED_COLUMN_FUNCTION_IS_NOT_ALLOWED > +create table t1 (a timestamp, b varchar(255) as (date_format(a, '%w %a %m %b')) stored); > diff --git a/mysql-test/suite/vcol/t/vcol_supported_sql_funcs_main.inc b/mysql-test/suite/vcol/t/vcol_supported_sql_funcs_main.inc > index dafc42098dd..fa2162e5981 100644 > --- a/mysql-test/suite/vcol/t/vcol_supported_sql_funcs_main.inc > +++ b/mysql-test/suite/vcol/t/vcol_supported_sql_funcs_main.inc > @@ -1197,6 +1197,12 @@ let $values1 = '2012-12-12',default; > let $rows = 1; > --source suite/vcol/inc/vcol_supported_sql_funcs.inc > > +--echo # DATE_FORMAT() STORED > +let $cols = a date, b varchar(100) as (date_format(a, '%W %a %M %b', 'de_DE')) STORED; > +let $values1 = '2012-12-12',default; > +let $rows = 1; > +--source suite/vcol/inc/vcol_supported_sql_funcs.inc > + > --echo # CURRENT_USER() > let $cols = a char, b varchar(32) as (current_user()); > let $values1 = 'a', default; > diff --git a/mysql-test/t/locale.test b/mysql-test/t/locale.test > index 4570b968ecf..a9a507bc387 100644 > --- a/mysql-test/t/locale.test > +++ b/mysql-test/t/locale.test > @@ -144,6 +144,8 @@ SET lc_time_names=@old_50915_lc_time_names; > > # Item::print > create view v1 as select > + date_format('2001-10-02', '%c %b %M') as a, > + date_format('2001-10-02', '%c %b %M', 'ru_RU') as a1, > format(123456789,2) as b, > format(123456789,2,'rm_CH') as b1; > select * from v1; > @@ -153,3 +155,29 @@ drop view v1; > --echo # > --echo # End of 10.2 tests > --echo # > + > +# > +# MDEV-11553 Can't restore a PERSISTENT column that uses DATE_FORMAT() > +# > +# 3-argument syntax for DATE_FORMAT() > +select date_format('2001-01-01', '%w %a %W', 'ro_RO'); > +select date_format('2001-01-03', '%w %a %W', 'ro_RO'); > +select date_format('2001-01-05', '%w %a %W', 'ro_RO'); > +select date_format('2001-01-07', '%w %a %W', 'ro_RO'); > +select date_format('2001-01-01', '%w %a %W', 'de_AT'); > +select date_format('2001-02-01', '%w %a %W', 'de_AT'); > +select date_format('2001-03-01', '%w %a %W', 'de_AT'); > +select date_format('2001-01-01', '%w %a %W', 'en_US'); > +select date_format('2001-03-01', '%c %b %M', 'en_US'); > +select date_format('2001-01-01', '%w %a %W', 'rm_CH'); > +select date_format('2001-01-03', '%w %a %W', 'rm_CH'); > +select date_format('2001-01-05', '%w %a %W', 'rm_CH'); > +select date_format('2001-01-07', '%w %a %W', 'rm_CH'); > +select date_format('2001-02-01', '%c %b %M', 'rm_CH'); > +select date_format('2001-04-01', '%c %b %M', 'rm_CH'); > +select date_format('2001-06-01', '%c %b %M', 'rm_CH'); > +select date_format('2001-08-01', '%c %b %M', 'rm_CH'); > +select date_format('2001-10-01', '%c %b %M', 'rm_CH'); > +select date_format('2001-12-01', '%c %b %M', 'rm_CH'); > +select date_format('2001-01-06', '%w %a %W', 'de_CH'); > +select date_format('2001-09-01', '%c %b %M', 'de_CH'); > diff --git a/sql/item_create.cc b/sql/item_create.cc > index e0bfe7a4402..88c1ac1f64f 100644 > --- a/sql/item_create.cc > +++ b/sql/item_create.cc > @@ -770,19 +770,6 @@ class Create_func_crosses : public Create_func_arg2 > #endif > > > -class Create_func_date_format : public Create_func_arg2 > -{ > -public: > - virtual Item *create_2_arg(THD *thd, Item *arg1, Item *arg2); > - > - static Create_func_date_format s_singleton; > - > -protected: > - Create_func_date_format() {} > - virtual ~Create_func_date_format() {} > -}; > - > - > class Create_func_datediff : public Create_func_arg2 > { > public: > @@ -4023,15 +4010,6 @@ Create_func_crosses::create_2_arg(THD *thd, Item *arg1, Item *arg2) > #endif > > > -Create_func_date_format Create_func_date_format::s_singleton; > - > -Item* > -Create_func_date_format::create_2_arg(THD *thd, Item *arg1, Item *arg2) > -{ > - return new (thd->mem_root) Item_func_date_format(thd, arg1, arg2, 0); > -} > - > - > Create_func_datediff Create_func_datediff::s_singleton; > > Item* > @@ -4530,7 +4508,7 @@ Create_func_from_unixtime::create_native(THD *thd, LEX_CSTRING *name, > Item *param_1= item_list->pop(); > Item *param_2= item_list->pop(); > Item *ut= new (thd->mem_root) Item_func_from_unixtime(thd, param_1); > - func= new (thd->mem_root) Item_func_date_format(thd, ut, param_2, 0); > + func= new (thd->mem_root) Item_func_date_format(thd, ut, param_2); > break; > } > default: > @@ -6483,7 +6461,7 @@ Create_func_time_format Create_func_time_format::s_singleton; > Item* > Create_func_time_format::create_2_arg(THD *thd, Item *arg1, Item *arg2) > { > - return new (thd->mem_root) Item_func_date_format(thd, arg1, arg2, 1); > + return new (thd->mem_root) Item_func_time_format(thd, arg1, arg2); > } > > > @@ -6844,7 +6822,6 @@ static Native_func_registry func_array[] = > { { C_STRING_WITH_LEN("CRC32") }, BUILDER(Create_func_crc32)}, > { { C_STRING_WITH_LEN("CROSSES") }, GEOM_BUILDER(Create_func_crosses)}, > { { C_STRING_WITH_LEN("DATEDIFF") }, BUILDER(Create_func_datediff)}, > - { { C_STRING_WITH_LEN("DATE_FORMAT") }, BUILDER(Create_func_date_format)}, > { { C_STRING_WITH_LEN("DAYNAME") }, BUILDER(Create_func_dayname)}, > { { C_STRING_WITH_LEN("DAYOFMONTH") }, BUILDER(Create_func_dayofmonth)}, > { { C_STRING_WITH_LEN("DAYOFWEEK") }, BUILDER(Create_func_dayofweek)}, > diff --git a/sql/item_strfunc.cc b/sql/item_strfunc.cc > index 83432f95612..3e12a545c97 100644 > --- a/sql/item_strfunc.cc > +++ b/sql/item_strfunc.cc > @@ -2668,11 +2668,11 @@ String *Item_func_soundex::val_str(String *str) > const int FORMAT_MAX_DECIMALS= 30; > > > -MY_LOCALE *Item_func_format::get_locale(Item *item) > +const MY_LOCALE *get_locale(Item *item) > { > - DBUG_ASSERT(arg_count == 3); > - String tmp, *locale_name= args[2]->val_str_ascii(&tmp); > - MY_LOCALE *lc; > + StringBuffer<MAX_FIELD_WIDTH> tmp; > + String *locale_name= item->val_str_ascii(&tmp); > + const MY_LOCALE *lc; > if (!locale_name || > !(lc= my_locale_by_name(locale_name->c_ptr_safe()))) > { > @@ -2712,7 +2712,7 @@ String *Item_func_format::val_str_ascii(String *str) > int dec; > /* Number of characters used to represent the decimals, including '.' */ > uint32 dec_length; > - MY_LOCALE *lc; > + const MY_LOCALE *lc; > DBUG_ASSERT(fixed == 1); > > dec= (int) args[1]->val_int(); > diff --git a/sql/item_strfunc.h b/sql/item_strfunc.h > index 685a64816be..542af13df5c 100644 > --- a/sql/item_strfunc.h > +++ b/sql/item_strfunc.h > @@ -28,6 +28,7 @@ > extern size_t username_char_length; > > class MY_LOCALE; > +const MY_LOCALE *get_locale(Item *item); > > class Item_str_func :public Item_func > { > @@ -932,14 +933,13 @@ class Item_func_make_set :public Item_str_func > > class Item_func_format :public Item_str_ascii_func > { > - MY_LOCALE *locale; > + const MY_LOCALE *locale; > public: > Item_func_format(THD *thd, Item *org, Item *dec): > Item_str_ascii_func(thd, org, dec) {} > Item_func_format(THD *thd, Item *org, Item *dec, Item *lang): > Item_str_ascii_func(thd, org, dec, lang) {} > > - MY_LOCALE *get_locale(Item *item); > String *val_str_ascii(String *); > void fix_length_and_dec(); > const char *func_name() const { return "format"; } > diff --git a/sql/item_timefunc.cc b/sql/item_timefunc.cc > index 86bde00cced..5e660c58583 100644 > --- a/sql/item_timefunc.cc > +++ b/sql/item_timefunc.cc > @@ -455,7 +455,8 @@ static bool extract_date_time(DATE_TIME_FORMAT *format, > */ > > static bool make_date_time(DATE_TIME_FORMAT *format, MYSQL_TIME *l_time, > - timestamp_type type, MY_LOCALE *locale, String *str) > + timestamp_type type, const MY_LOCALE *locale, > + String *str) > { > char intbuff[15]; > uint hours_i; > @@ -1833,7 +1834,14 @@ bool Item_func_sec_to_time::get_date(MYSQL_TIME *ltime, ulonglong fuzzy_date) > void Item_func_date_format::fix_length_and_dec() > { > THD* thd= current_thd; > - locale= thd->variables.lc_time_names; > + if (!is_time_format) > + { > + if (arg_count < 3) > + locale= thd->variables.lc_time_names; > + else > + if (args[2]->basic_const_item()) > + locale= get_locale(args[2]); > + } > > /* > Must use this_item() in case it's a local SP variable > @@ -1875,6 +1883,8 @@ bool Item_func_date_format::eq(const Item *item, bool binary_cmp) const > if (this == item) > return 1; > item_func= (Item_func_date_format*) item; > + if (arg_count != item_func->arg_count) > + return 0; > if (!args[0]->eq(item_func->args[0], binary_cmp)) > return 0; > /* > @@ -1884,6 +1894,8 @@ bool Item_func_date_format::eq(const Item *item, bool binary_cmp) const > */ > if (!args[1]->eq(item_func->args[1], 1)) > return 0; > + if (arg_count > 2 && !args[2]->eq(item_func->args[2], 1)) > + return 0; > return 1; > } > > @@ -1967,15 +1979,18 @@ String *Item_func_date_format::val_str(String *str) > String *format; > MYSQL_TIME l_time; > uint size; > - int is_time_flag = is_time_format ? TIME_TIME_ONLY : 0; > + const MY_LOCALE *lc= 0; > DBUG_ASSERT(fixed == 1); > > - if (get_arg0_date(&l_time, is_time_flag)) > + if (get_arg0_date(&l_time, is_time_format ? TIME_TIME_ONLY : 0)) > return 0; > > if (!(format = args[1]->val_str(str)) || !format->length()) > goto null_date; > > + if (!is_time_format && !(lc= locale) && !(lc= get_locale(args[2]))) > + goto null_date; // invalid locale > + > if (fixed_length) > size=max_length; > else > @@ -1998,7 +2013,7 @@ String *Item_func_date_format::val_str(String *str) > if (!make_date_time(&date_time_format, &l_time, > is_time_format ? MYSQL_TIMESTAMP_TIME : > MYSQL_TIMESTAMP_DATE, > - locale, str)) > + lc, str)) > return str; > > null_date: > diff --git a/sql/item_timefunc.h b/sql/item_timefunc.h > index 7d2fe46b644..20ecb4774b3 100644 > --- a/sql/item_timefunc.h > +++ b/sql/item_timefunc.h > @@ -797,22 +797,24 @@ class Item_func_from_days :public Item_datefunc > > class Item_func_date_format :public Item_str_func > { > - MY_LOCALE *locale; > + const MY_LOCALE *locale; > int fixed_length; > - const bool is_time_format; > String value; > +protected: > + bool is_time_format; > public: > - Item_func_date_format(THD *thd, Item *a, Item *b, bool is_time_format_arg): > - Item_str_func(thd, a, b), is_time_format(is_time_format_arg) {} > + Item_func_date_format(THD *thd, Item *a, Item *b): > + Item_str_func(thd, a, b), locale(0), is_time_format(false) {} > + Item_func_date_format(THD *thd, Item *a, Item *b, Item *c): > + Item_str_func(thd, a, b, c), locale(0), is_time_format(false) {} > String *val_str(String *str); > - const char *func_name() const > - { return is_time_format ? "time_format" : "date_format"; } > + const char *func_name() const { return "date_format"; } > void fix_length_and_dec(); > uint format_length(const String *format); > bool eq(const Item *item, bool binary_cmp) const; > bool check_vcol_func_processor(void *arg) > { > - if (is_time_format) > + if (arg_count > 2) > return false; > return mark_unsupported_function(func_name(), "()", arg, VCOL_SESSION_FUNC); > } > @@ -820,6 +822,17 @@ class Item_func_date_format :public Item_str_func > { return get_item_copy<Item_func_date_format>(thd, mem_root, this); } > }; > > +class Item_func_time_format: public Item_func_date_format > +{ > +public: > + Item_func_time_format(THD *thd, Item *a, Item *b): > + Item_func_date_format(thd, a, b) { is_time_format= true; } > + const char *func_name() const { return "time_format"; } > + bool check_vcol_func_processor(void *arg) { return false; } > + Item *get_copy(THD *thd, MEM_ROOT *mem_root) > + { return get_item_copy<Item_func_time_format>(thd, mem_root, this); } > +}; > + > > class Item_func_from_unixtime :public Item_datetimefunc > { > diff --git a/sql/lex.h b/sql/lex.h > index f2dac95fce5..67c3bc8620d 100644 > --- a/sql/lex.h > +++ b/sql/lex.h > @@ -726,6 +726,7 @@ static SYMBOL sql_functions[] = { > { "CURTIME", SYM(CURTIME)}, > { "DATE_ADD", SYM(DATE_ADD_INTERVAL)}, > { "DATE_SUB", SYM(DATE_SUB_INTERVAL)}, > + { "DATE_FORMAT", SYM(DATE_FORMAT_SYM)}, > { "DECODE", SYM(DECODE_SYM)}, > { "DENSE_RANK", SYM(DENSE_RANK_SYM)}, > { "EXTRACT", SYM(EXTRACT_SYM)}, > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index 7e89ad91a17..17d6f016b2c 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -875,7 +875,7 @@ bool my_yyoverflow(short **a, YYSTYPE **b, ulong *yystacksize); > Comments for TOKENS. > For each token, please include in the same line a comment that contains > the following tags: > - SQL-2011-N : Non Reserved keywird as per SQL-2011 > + SQL-2011-N : Non Reserved keyword as per SQL-2011 > SQL-2003-R : Reserved keyword as per SQL-2003 > SQL-2003-N : Non Reserved keyword as per SQL-2003 > SQL-1999-R : Reserved keyword as per SQL-1999 > @@ -1010,6 +1010,7 @@ bool my_yyoverflow(short **a, YYSTYPE **b, ulong *yystacksize); > %token DATA_SYM /* SQL-2003-N */ > %token DATETIME > %token DATE_ADD_INTERVAL /* MYSQL-FUNC */ > +%token DATE_FORMAT_SYM /* MYSQL-FUNC */ > %token DATE_SUB_INTERVAL /* MYSQL-FUNC */ > %token DATE_SYM /* SQL-2003-R */ > %token DAY_HOUR_SYM > @@ -9806,6 +9807,18 @@ function_call_nonkeyword: > if ($$ == NULL) > MYSQL_YYABORT; > } > + | DATE_FORMAT_SYM '(' expr ',' expr ')' > + { > + $$= new (thd->mem_root) Item_func_date_format(thd, $3, $5); > + if ($$ == NULL) > + MYSQL_YYABORT; > + } > + | DATE_FORMAT_SYM '(' expr ',' expr ',' expr ')' > + { > + $$= new (thd->mem_root) Item_func_date_format(thd, $3, $5, $7); > + if ($$ == NULL) > + MYSQL_YYABORT; > + } > | DECODE_SYM '(' expr ',' expr ')' > { > $$= new (thd->mem_root) Item_func_decode(thd, $3, $5); > @@ -14719,6 +14732,7 @@ keyword_sp_not_data_type: > | CYCLE_SYM {} > | DATA_SYM {} > | DATAFILE_SYM {} > + | DATE_FORMAT_SYM {} > | DAY_SYM {} > | DECODE_SYM {} > | DEFINER_SYM {} > diff --git a/sql/sql_yacc_ora.yy b/sql/sql_yacc_ora.yy > index a475914d369..54ddd3ae922 100644 > --- a/sql/sql_yacc_ora.yy > +++ b/sql/sql_yacc_ora.yy > @@ -284,7 +284,7 @@ bool my_yyoverflow(short **a, YYSTYPE **b, ulong *yystacksize); > Comments for TOKENS. > For each token, please include in the same line a comment that contains > the following tags: > - SQL-2011-N : Non Reserved keywird as per SQL-2011 > + SQL-2011-N : Non Reserved keyword as per SQL-2011 > SQL-2003-R : Reserved keyword as per SQL-2003 > SQL-2003-N : Non Reserved keyword as per SQL-2003 > SQL-1999-R : Reserved keyword as per SQL-1999 > @@ -419,6 +419,7 @@ bool my_yyoverflow(short **a, YYSTYPE **b, ulong *yystacksize); > %token DATA_SYM /* SQL-2003-N */ > %token DATETIME > %token DATE_ADD_INTERVAL /* MYSQL-FUNC */ > +%token DATE_FORMAT_SYM /* MYSQL-FUNC */ > %token DATE_SUB_INTERVAL /* MYSQL-FUNC */ > %token DATE_SYM /* SQL-2003-R */ > %token DAY_HOUR_SYM > @@ -9462,6 +9463,18 @@ column_default_non_parenthesized_expr: > if ($$ == NULL) > MYSQL_YYABORT; > } > + | DATE_FORMAT_SYM '(' expr ',' expr ')' > + { > + $$= new (thd->mem_root) Item_func_date_format(thd, $3, $5); > + if ($$ == NULL) > + MYSQL_YYABORT; > + } > + | DATE_FORMAT_SYM '(' expr ',' expr ',' expr ')' > + { > + $$= new (thd->mem_root) Item_func_date_format(thd, $3, $5, $7); > + if ($$ == NULL) > + MYSQL_YYABORT; > + } > | DECODE_SYM '(' expr ',' decode_when_list ')' > { > $5->push_front($3, thd->mem_root); > @@ -14893,6 +14906,7 @@ keyword_sp_not_data_type: > | CYCLE_SYM {} > | DATA_SYM {} > | DATAFILE_SYM {} > + | DATE_FORMAT_SYM {} > | DAY_SYM {} > | DECODE_SYM {} > | DEFINER_SYM {} > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits >

1 0

Re: [Maria-developers] [Commits] c132349bc79: MDEV-11846: ERROR 1114 (HY000) table full when performing GROUP BY
by Sergey Petrunia 21 Sep '17

21 Sep '17

Hi Varun, Ok to push. On Fri, Sep 15, 2017 at 01:28:53PM +0530, Varun wrote: > revision-id: c132349bc79d987947c8659e71601914cfec71e8 (mariadb-10.1.23-156-gc132349bc79) > parent(s): fa2701c6f7b028782cf231565f578b2fc0f10d51 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2017-09-15 13:27:11 +0530 > message: > > MDEV-11846: ERROR 1114 (HY000) table full when performing GROUP BY > > The problem is there is an overflow for the key_file_length. > Added the maximum limit for the key_file_length > > --- > storage/maria/ma_create.c | 41 +++++++++++++++++++++++++++++++++-------- > 1 file changed, 33 insertions(+), 8 deletions(-) > > diff --git a/storage/maria/ma_create.c b/storage/maria/ma_create.c > index 0ddd8b226e2..79e60a76abc 100644 > --- a/storage/maria/ma_create.c > +++ b/storage/maria/ma_create.c > @@ -660,11 +660,24 @@ int maria_create(const char *name, enum data_file_type datafile_type, > > if (length > max_key_length) > max_key_length= length; > - tot_length+= ((max_rows/(ulong) (((uint) maria_block_size - > + > + if (tot_length == ULLONG_MAX) > + continue; > + > + ulonglong tot_length_part= (max_rows/(ulong) (((uint) maria_block_size - > MAX_KEYPAGE_HEADER_SIZE - > KEYPAGE_CHECKSUM_SIZE)/ > - (length*2))) * > - maria_block_size); > + (length*2))); > + if (tot_length_part >= (ULLONG_MAX / maria_block_size + > + ULLONG_MAX % maria_block_size)) > + tot_length= ULLONG_MAX; > + else > + { > + if (tot_length > ULLONG_MAX - tot_length_part * maria_block_size) > + tot_length= ULLONG_MAX; > + else > + tot_length+= tot_length_part * maria_block_size; > + } > } > > unique_key_parts=0; > @@ -673,11 +686,24 @@ int maria_create(const char *name, enum data_file_type datafile_type, > uniquedef->key=keys+i; > unique_key_parts+=uniquedef->keysegs; > share.state.key_root[keys+i]= HA_OFFSET_ERROR; > - tot_length+= (max_rows/(ulong) (((uint) maria_block_size - > + > + if (tot_length == ULLONG_MAX) > + continue; > + ulonglong tot_length_part= (max_rows/(ulong) (((uint) maria_block_size - > MAX_KEYPAGE_HEADER_SIZE - > KEYPAGE_CHECKSUM_SIZE) / > - ((MARIA_UNIQUE_HASH_LENGTH + pointer)*2)))* > - (ulong) maria_block_size; > + ((MARIA_UNIQUE_HASH_LENGTH + pointer)*2))); > + > + if (tot_length_part >= (ULLONG_MAX / maria_block_size + > + ULLONG_MAX % maria_block_size)) > + tot_length= ULLONG_MAX; > + else > + { > + if (tot_length > ULLONG_MAX - tot_length_part * maria_block_size) > + tot_length= ULLONG_MAX; > + else > + tot_length+= tot_length_part * maria_block_size; > + } > } > keys+=uniques; /* Each unique has 1 key */ > key_segs+=uniques; /* Each unique has 1 key seg */ > @@ -746,8 +772,7 @@ int maria_create(const char *name, enum data_file_type datafile_type, > Get estimate for index file length (this may be wrong for FT keys) > This is used for pointers to other key pages. > */ > - tmp= (tot_length + maria_block_size * keys * > - MARIA_INDEX_BLOCK_MARGIN) / maria_block_size; > + tmp= (tot_length / maria_block_size + keys * MARIA_INDEX_BLOCK_MARGIN); > > /* > use maximum of key_file_length we calculated and key_file_length value we > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] 0319ee58fbb: MDEV-13290: Assertion Assertion `!is_set() || (m_status == DA_OK_BULK && is_bulk_op())' or `! is_set()' failed
by Alexander Barkov 19 Sep '17

19 Sep '17

Hi Sanja, The patch looks OK for me. Please find one suggestion below: On 09/19/2017 05:56 PM, Oleksandr Byelkin wrote: > revision-id: 0319ee58fbb6372ccc8533613e0c55666abc948a (mariadb-10.0.32-32-g0319ee58fbb) > parent(s): 389f7cdf3ccd11a6b4fb9b6346436790254e615c > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2017-09-19 15:55:59 +0200 > message: > > MDEV-13290: Assertion Assertion `!is_set() || (m_status == DA_OK_BULK && is_bulk_op())' or `! is_set()' failed > > Check error status which can be set by conversion procedures. > > --- > mysql-test/r/insert.result | 22 ++++++++++++++++++++++ > mysql-test/t/insert.test | 26 ++++++++++++++++++++++++++ > sql/table.cc | 3 +++ > 3 files changed, 51 insertions(+) > > diff --git a/mysql-test/r/insert.result b/mysql-test/r/insert.result > index 82f3977e231..0efee36e22d 100644 > --- a/mysql-test/r/insert.result > +++ b/mysql-test/r/insert.result > @@ -717,3 +717,25 @@ insert ignore into t1 values (1,12); > Warnings: > Warning 1062 Duplicate entry '1' for key 'f1' > DROP TABLE t1; > +# > +# MDEV-13290 Assertion Assertion `!is_set() || (m_status == DA_OK_BULK > +# && is_bulk_op())' or `! is_set()' failed > +# > +SET @save_mode= @@sql_mode; > +SET sql_mode= 'STRICT_ALL_TABLES'; > +CREATE TABLE t1 (f1 INT DEFAULT 0, f2 INT); > +CREATE ALGORITHM = MERGE VIEW v1 AS SELECT f1, f2 FROM t1 WHERE f1 = 'x' WITH CHECK OPTION; > +REPLACE INTO v1 SET f2 = 1; > +ERROR 22007: Truncated incorrect DOUBLE value: 'x' > +drop view v1; > +CREATE ALGORITHM = MERGE VIEW v1 AS SELECT f1, f2 FROM t1 WHERE f1 = cast('' as decimal) WITH CHECK OPTION; > +REPLACE INTO v1 SET f2 = 1; > +ERROR 22007: Truncated incorrect DECIMAL value: '' > +drop view v1; > +SELECT 0,0 INTO OUTFILE 't1.txt'; > +CREATE ALGORITHM = MERGE VIEW v1 AS SELECT f1, f2 FROM t1 WHERE f1 = 'x' WITH CHECK OPTION; > +LOAD DATA INFILE 't1.txt' INTO TABLE v1; > +ERROR 22007: Truncated incorrect DOUBLE value: 'x' > +drop view v1; > +drop table t1; > +SET @@sql_mode= @save_mode; > diff --git a/mysql-test/t/insert.test b/mysql-test/t/insert.test > index ff8396fd7fd..d84c0cee90d 100644 > --- a/mysql-test/t/insert.test > +++ b/mysql-test/t/insert.test > @@ -573,3 +573,29 @@ insert ignore into t1 values (1,12) on duplicate key update f2=13; > set @@old_mode=""; > insert ignore into t1 values (1,12); > DROP TABLE t1; > + > +--echo # > +--echo # MDEV-13290 Assertion Assertion `!is_set() || (m_status == DA_OK_BULK > +--echo # && is_bulk_op())' or `! is_set()' failed > +--echo # > + > +SET @save_mode= @@sql_mode; > +SET sql_mode= 'STRICT_ALL_TABLES'; > +CREATE TABLE t1 (f1 INT DEFAULT 0, f2 INT); > +CREATE ALGORITHM = MERGE VIEW v1 AS SELECT f1, f2 FROM t1 WHERE f1 = 'x' WITH CHECK OPTION; > +--error ER_TRUNCATED_WRONG_VALUE > +REPLACE INTO v1 SET f2 = 1; Can you please add "SELECT * FROM t1" here and some other places (see below), to make sure that REPLACE did not actually insert new records. Please see my comments in MDEV. > +drop view v1; > +CREATE ALGORITHM = MERGE VIEW v1 AS SELECT f1, f2 FROM t1 WHERE f1 = cast('' as decimal) WITH CHECK OPTION; > +--error ER_TRUNCATED_WRONG_VALUE > +REPLACE INTO v1 SET f2 = 1; Please add "SELECT * FROM t1". > +drop view v1; > +SELECT 0,0 INTO OUTFILE 't1.txt'; > +CREATE ALGORITHM = MERGE VIEW v1 AS SELECT f1, f2 FROM t1 WHERE f1 = 'x' WITH CHECK OPTION; > +--error ER_TRUNCATED_WRONG_VALUE > +LOAD DATA INFILE 't1.txt' INTO TABLE v1; Please add "SELECT * FROM t1". > +let $MYSQLD_DATADIR= `select @@datadir`; > +remove_file $MYSQLD_DATADIR/test/t1.txt; > +drop view v1; > +drop table t1; > +SET @@sql_mode= @save_mode; > diff --git a/sql/table.cc b/sql/table.cc > index 975d9d53882..ff9c4217b7d 100644 > --- a/sql/table.cc > +++ b/sql/table.cc > @@ -4642,6 +4642,9 @@ int TABLE_LIST::view_check_option(THD *thd, bool ignore_failure) > main_view->view_name.str); > return(VIEW_CHECK_ERROR); > } > + /* We check thd->error() because it can be set by conversion problem. */ > + if (thd->is_error()) > + return(VIEW_CHECK_ERROR); > return(VIEW_CHECK_OK); > } > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits >

1 0

[Maria-developers] Patch for MDEV-10574
by jerome brauge 19 Sep '17

19 Sep '17

Alexander, Here is the patch I was talking about. It not address all aspects of MDEV-10574, especially, empty strings in table column are not view as null. But as in oracle mode, no empty string in columns should be created, an fully oracle application will never fall in this case. Regards, Jérôme. > -----Message d'origine----- > De : jerome brauge > Envoyé : mardi 19 septembre 2017 14:13 > À : 'Alexander Barkov' > Objet : RE: Oracle strings functions compatibility : substr > > Hello Alexander, > Yes I'm aware. > This point will be treated by my next patch which will also affect the others > string functions (a part of MDEV-10574). > I will refresh this new patch with the current development branch, and I > submit to you. > Regards. > Jérôme. > > > -----Message d'origine----- > > De : Alexander Barkov [mailto:bar@mariadb.org] Envoyé : mardi 19 > > septembre 2017 13:16 À : jerome brauge Objet : Re: Oracle strings > > functions compatibility : substr > > > > Hi Jerome, > > > > I noticed one more difference in SUBSTR: > > > > > > SELECT SUBSTR('aaa',1,-1) FROM DUAL; > > > > MariaDB returns an empty string. > > Oracle returns NULL. > > > > Here's Oracle's documentation: > > > > https://docs.oracle.com/cd/B19306_01/server.102/b14200/functions162.ht > > m > > > > > > I think we should do this in the same patch, to avoid behavior change > > in the future. > > > > Would you like to try doing this additional change? > > Or would you like me to make an incremental patch on top of yours? > > > > Thanks! > > > > On 09/19/2017 10:33 AM, jerome brauge wrote: > > > Hello Alexander, > > > It's done. > > > > > > Do you have news from Sergei for MDEV-12874, MDEV-13417 and MDEV- > > 13418 ? > > > > I asked Sergei, awaiting for his answer. I'll let you know. > > > > > > > > Thank you very much. > > > > > > > > > > > >> -----Message d'origine----- > > >> De : Alexander Barkov [mailto:bar@mariadb.org] Envoyé : lundi 18 > > >> septembre 2017 18:12 À : jerome brauge Objet : Re: Oracle strings > > >> functions compatibility : substr > > >> > > >> Hello Jerome, > > >> > > >> > > >> On 09/18/2017 05:14 PM, jerome brauge wrote: > > >>> Hello Alexander, > > >>> I don't understand how I've missed such an obvious solution ! > > >>> Here is the new patch. > > >> > > >> Thanks. Now it looks simpler. > > >> > > >> Can you please do one small thing: > > >> move get_position() from "public:" to "protected:". > > >> > > >> Ok to make a pull request after this change. > > >> > > >> Thank you very much! > > >> > > >> > > >>> > > >>> Regards. > > >>> Jérôme. > > >>> > > >>> > > >>>> -----Message d'origine----- > > >>>> De : Alexander Barkov [mailto:bar@mariadb.org] Envoyé : lundi 18 > > >>>> septembre 2017 11:30 À : jerome brauge Objet : Re: Oracle strings > > >>>> functions compatibility : substr > > >>>> > > >>>> Hello Jerome, > > >>>> > > >>>> > > >>>> On 08/17/2017 04:43 PM, jerome brauge wrote: > > >>>>> Hello Alexander, > > >>>>> Here is a patch for function SUBSTR in sql_mode=oracle (If > > >>>>> position is 0, > > >>>> then it is treated as 1). > > >>>> > > >>>> I'm thinking of how to get a smaller patch. > > >>>> > > >>>> Wouldn't it be possible to introduce: > > >>>> > > >>>> virtual longlong get_position(); > > >>>> > > >>>> > > >>>> > > >>>> longlong Item_func_substr::get_position() { > > >>>> return args[1]->val_int(); > > >>>> } > > >>>> > > >>>> longlong Item_func_substr_oracle::get_position() > > >>>> { > > >>>> longlong pos= args[1]->val_int(); > > >>>> return pos == 0 ? 1 : pos; > > >>>> } > > >>>> > > >>>> > > >>>> and just replace all calls for args[1]->val_int() to get_position() ? > > >>>> > > >>>> Thanks! > > >>>> > > >>>> > > >>>>> > > >>>>> Regards, > > >>>>> Jérôme. > > >>>>>

1 0

Re: [Maria-developers] NUMA Support in MariaDB
by serg＠mariadb.org 19 Sep '17

19 Sep '17

Hi, Sumit! On Sep 16, Sumit Lakra wrote: > The project 'NUMA Support in MariaDB' is still incomplete and like I > mentioned in my mails to my mentors, I am still willing to work on it > and see it through to the end. Great, thanks! > Most of the minor tasks along with some important tasks are already > complete. The important part that remains is to work on the task-queue > interface between SQL and InnoDB. I was unable to implement this for > the NUMA case and I won't be able to complete it without some help. > So, do you think you can connect me with someone from the > MariaDB/InnoDB/MySQL team who is an expert of this interface? I will > need someone who has a very good understanding of the present > structure to help me come up with a way to implement this for the NUMA > case. If you do find someone who can be of help, please see to it that > since GSoC is over (and otherwise as well), it shouldn't be a > Student-Mentor kind of thing, but more like two or more developers > working together trying to come up with a solution to this problem. Why wouldn't you ask the question on the maria-developers@ mailing list? I'll try to make sure that it'll be answered. > However, if you are unable to connect me with someone who has > expertise in the handler interface, I am afraid I won't be able to > complete this project on my own. In this case, would you like me to > create a pull request of the work done so far ( > https://github.com/theGodlessLakra/server/tree/NUMA_Support) Although > NUMA Support is incomplete, merging this code to the MariaDB code base > will allow other developers to continue working on this. Is it sufficiently complete to be useful as is? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 1