- developers - lists.mariadb.org

[Maria-developers] Open and close a TABLE_LIST at random place in code
by Eugene Kosov 31 May '18

31 May '18

Hello. I need to see a table structure while executing some query (DELETE in my case, but there will be more query types in future). I have a TABLE_LIST which came from a parser. Now I'm opening it with open_table_only_view_structure(), examine table structure and close it, trying to prevent any side effects. And there is a lot of them. Current code looks like this: if (open_tables_only_view_structure(thd, table_list, false)) { return true; } // examine table structure close_mysql_tables(thd); for (Sroutine_hash_entry *e = thd->lex->sroutines_list.first;e; e=e->next) e->mdl_request.ticket= NULL; for (TABLE_LIST *it= table_list; it; it= it->next_global) { it->table= NULL; it->reinit_before_use(thd); //it->next_global= NULL; it->derived_result= NULL; it->derived= NULL; it->view= NULL; it->view_tables= NULL; it->view_sctx= NULL; it->file_version= 0; it->mariadb_version= 0; it->updatable_view= 0; it->view_suid= 0; it->derived_type= 0; it->open_type= OT_BASE_ONLY; it->view_creation_ctx= 0; // it->set_table_ref_id(TABLE_REF_NULL, 0); } Is there a way to close a TABLE_LIST gracefully without touching a lot of it's fields? Or even maybe it's possible to open a table in a simpler manner? -- Eugene

2 3

Re: [Maria-developers] [Commits] de722585c1b: MDEV-16225: wrong resultset from query with semijoin=on
by Sergey Petrunia 27 May '18

27 May '18

Hi Varun, On Thu, May 24, 2018 at 03:17:02AM +0530, Varun wrote: > revision-id: de722585c1b2ad2af99bd1749e8566ea77838764 (mariadb-5.5.56-219-gde722585c1b) > parent(s): 1ada4afb0a51f7283b6187a95019ec2cb80c8a0b > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-05-24 03:09:42 +0530 > message: > > MDEV-16225: wrong resultset from query with semijoin=on > > For non-semi-join subquery optimization we do a cost based decision between > Materialisation and IN -> EXIST transformation. The issue in this case is that for IN->EXIST transformation > we run JOIN::reoptimize with the IN->EXISt conditions and we come up with a new query plan. But when we compare > the cost with Materialization, we make the decision to chose Materialization so we need to restore the query plan > for Materilization. The restoring of query plan is not handled correctly. > > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index d6d269a700f..b8d43b9797e 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -23805,6 +23805,22 @@ void JOIN::restore_query_plan(Join_plan_state *restore_from) > } > > } > + else > + { > + /* > + After running JOIN::reoptimize for IN->EXISTS > + transformation we might add elements to the keyuse array, > + The original query plan was that the keyuse array had no elements, so we > + should restore that state here. > + */ > + while(keyuse.elements) > + delete_dynamic_element(&keyuse, 0); I'm wondering, why do we need the above loop and not just delete everything with delete_dynamic(). But if I don't miss anything, this is not significant. Ok to push either the above variant or one with delete_dynamic(). > + for (uint i= 0; i < table_count; i++) > + { > + join_tab[i].keyuse= NULL; > + join_tab[i].checked_keys.clear_all(); > + } > + } > memcpy((uchar*) best_positions, (uchar*) restore_from->best_positions, > sizeof(POSITION) * (table_count + 1)); > /* Restore SJM nests */ > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Maria-discuss] Query ambiguity
by Karthick Subramanian 17 May '18

17 May '18

Check whether you really need a comma here: *metrics_5.id <http://metrics_5.id/>,* *(SELECT * Also in your screen shot, AS 'LT CPC`, there is no comma and you use derived table in SELECT and trying to use JOIN in the FROM using this. Sorry to say this, I think you can either break this logic into small simple chunks and use SP to derive the logic in case you are not much comfortable writing a complex SQL. I think there is lot of syntax errors on it. On Thu, May 17, 2018 at 10:05 AM, Dev C <chauhan.devsur(a)gmail.com> wrote: > Here is the full code and picture on what is required. > > SELECT > campaigns_5.name AS `Campaign Name`, > adgroups_5.name AS `Adgroup Name`, > STNegData1W.query, > keywords_5.matchType AS `Match Type`, > Sum(metrics_5.clicks) AS `LT Clicks`, > Sum(metrics_5.costs) AS `LT Spend`, > Sum(metrics_5.orders) AS `LT Orders`, > metrics_5.sales AS `LT Sales`, > If(Sum(metrics_5.sales)=0,0,Sum(metrics_5.costs)/Sum(metrics_5.sales)) > AS `LT ACOS`, > If(Sum(clicks)=0,0,Sum(costs)/Sum(clicks)) AS `LT CPC` > FROM (((keywords_5 INNER JOIN campaigns_5 ON keywords_5.campaignId = > campaigns_5.campaignId) INNER JOIN adgroups_5 ON keywords_5.adGroupId = > adgroups_5.adGroupId) INNER JOIN STNegData1W ON keywords_5.keywordId = > STNegData1W.keywordId) INNER JOIN metrics_5 ON STNegData1W.metricId = *metrics_5.id > <http://metrics_5.id>,* > *(SELECT * > STNegData4W.query, > STNegData4W.metricId, > STNegData4W.keywordId, > metrics_5.impressions > FROM > (SELECT searchterm_5.query, searchterm_5.metricId, > searchterm_5.keywordId, Sum(metrics_5.orders) AS `4W Orders` > FROM searchterm_5 INNER JOIN metrics_5 ON searchterm_5.metricId = > metrics_5.id > GROUP BY searchterm_5.query, searchterm_5.metricId, > searchterm_5.keywordId, metrics_5.created_at > HAVING (((Sum(metrics_5.orders))=0) AND ((metrics_5.created_at) > > DATE_SUB('2018-05-10',INTERVAL 28 DAY)))) AS STNegData4W > INNER JOIN > metrics_5 ON STNegData4W.metricId = metrics_5.id > GROUP BY STNegData4W.query, STNegData4W.metricId, STNegData4W.keywordId, > metrics_5.impressions, metrics_5.created_at > HAVING (((metrics_5.impressions)>0) AND ((metrics_5.created_at)> > DATE_SUB('2018-05-10', INTERVAL 7 DAY)))) AS STNegData1W > GROUP BY campaigns_5.name, adgroups_5.name, STNegData1W.query, > keywords_5.matchType, metrics_5.sales, campaigns_5.user_market_id, Right( > campaigns_5.name,4) > HAVING (((Sum(metrics_5.clicks))>5) AND ((campaigns_5.user_market_id)=12) > AND ((Right(campaigns_5.name,4))="PFUS")) > > On Thu, May 17, 2018 at 4:06 PM, Pantelis Theodosiou <ypercube(a)gmail.com> > wrote: > >> What Rhys said, please post queries as text, not as images. >> >> The issue is likely due to the way you structured your query. You have >> put a derived table in a random place (in the SELECT list). >> >> Please try to read about CTEs. It will help you design those complex >> queries better and make them more readable and easy to edit: >> https://mariadb.com/kb/en/library/with/ >> >> Best regards >> >> Pantelis Theodosiou >> >> On Thu, May 17, 2018 at 11:23 AM, <Rhys.Campbell(a)swisscom.com> wrote: >> >>> You're not making a lot of sense here. Please explain the "ambiguity". >>> That's a fairly complex query so you should include as text. >>> >>> >>> >>> "Does not work" <- In the image. It helps if you include error message. >>> >>> >>> >>> *From:* Maria-discuss [mailto:maria-discuss-bounces+rhys.campbell= >>> swisscom.com(a)lists.launchpad.net] *On Behalf Of *Dev C >>> *Sent:* 17 May 2018 06:41 >>> *To:* maria-discuss(a)lists.launchpad.net; Maria Developers < >>> maria-developers(a)lists.launchpad.net> >>> *Subject:* [Maria-discuss] Query ambiguity >>> >>> >>> >>> Hello >>> >>> >>> >>> I have query ambiguity as attached. Please help me on how to solve that. >>> >>> >>> >>> >>> _______________________________________________ >>> Mailing list: https://launchpad.net/~maria-discuss >>> Post to : maria-discuss(a)lists.launchpad.net >>> Unsubscribe : https://launchpad.net/~maria-discuss >>> More help : https://help.launchpad.net/ListHelp >>> >>> >> > > _______________________________________________ > Mailing list: https://launchpad.net/~maria-discuss > Post to : maria-discuss(a)lists.launchpad.net > Unsubscribe : https://launchpad.net/~maria-discuss > More help : https://help.launchpad.net/ListHelp > >

1 0

Re: [Maria-developers] [Maria-discuss] Query ambiguity
by Pantelis Theodosiou 17 May '18

17 May '18

What Rhys said, please post queries as text, not as images. The issue is likely due to the way you structured your query. You have put a derived table in a random place (in the SELECT list). Please try to read about CTEs. It will help you design those complex queries better and make them more readable and easy to edit: https://mariadb.com/kb/en/library/with/ Best regards Pantelis Theodosiou On Thu, May 17, 2018 at 11:23 AM, <Rhys.Campbell(a)swisscom.com> wrote: > You're not making a lot of sense here. Please explain the "ambiguity". > That's a fairly complex query so you should include as text. > > > > "Does not work" <- In the image. It helps if you include error message. > > > > *From:* Maria-discuss [mailto:maria-discuss-bounces+rhys.campbell= > swisscom.com(a)lists.launchpad.net] *On Behalf Of *Dev C > *Sent:* 17 May 2018 06:41 > *To:* maria-discuss(a)lists.launchpad.net; Maria Developers < > maria-developers(a)lists.launchpad.net> > *Subject:* [Maria-discuss] Query ambiguity > > > > Hello > > > > I have query ambiguity as attached. Please help me on how to solve that. > > > > > _______________________________________________ > Mailing list: https://launchpad.net/~maria-discuss > Post to : maria-discuss(a)lists.launchpad.net > Unsubscribe : https://launchpad.net/~maria-discuss > More help : https://help.launchpad.net/ListHelp > >

2 1

Re: [Maria-developers] [Maria-discuss] Interval in MYsql query
by Peter Laursen 16 May '18

16 May '18

Use the DATE_ADD( ) or DATE_SUB() function with INTERVAL as explained here: https://dev.mysql.com/doc/refman/8.0/en/date-and-time-functions.html#functi… -- Peter. On Wed, May 16, 2018 at 4:07 PM, Dev C <chauhan.devsur(a)gmail.com> wrote: > Hello > > If I use below query > (metrics_5.created_at)>'5/10/2018'- 28 ) > it gives me some result > but if i use below query > (metrics_5.created_at)>'5/10/2018'- INTERVAL 28 DAY ) > > It does not give me any result, any one know why ? > > _______________________________________________ > Mailing list: https://launchpad.net/~maria-discuss > Post to : maria-discuss(a)lists.launchpad.net > Unsubscribe : https://launchpad.net/~maria-discuss > More help : https://help.launchpad.net/ListHelp > >

1 0

[Maria-developers] [question] password input length
by Michal Schorm 16 May '18

16 May '18

Hello everyone, 1) I found out, that passwords longer than 79 characters aren't accepted via password prompt. Is that expected? Morie info and reproducer in RH bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1566179 -- 2) Another password length cap at 213 characters can be found when using sha256_password authentification plugin. Is that expected too? Morie info and reproducer in RH bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1566219 -- 3) Different theme: why aren't tests for mariadb-connector-odbc, that can be found on GitHub, packged to the source tarball ? -- Michal Schorm Associate Software Engineer Core Services - Databases Team Red Hat

2 2

Re: [Maria-developers] a639eff: MDEV-15813 ASAN use-after-poison in hp_hashnr upon HANDLER READ on a versioned HEAP table.
by Sergei Golubchik 16 May '18

16 May '18

Hi, Alexey! On May 16, Alexey Botchkov wrote: > revision-id: a639eff5940ed4147709511aeaee922fa7095284 (mariadb-10.3.6-193-ga639eff) > parent(s): cd15e764a8a4f3e799f1efaede413c95e3233df6 > committer: Alexey Botchkov > timestamp: 2018-05-16 09:44:22 +0400 > message: > > MDEV-15813 ASAN use-after-poison in hp_hashnr upon HANDLER READ on a versioned HEAP table. > > Check index capabilities before executing HANDLER READ command. > > diff --git a/sql/sql_handler.cc b/sql/sql_handler.cc > index 187a746..02d5bbf 100644 > --- a/sql/sql_handler.cc > +++ b/sql/sql_handler.cc > @@ -660,6 +660,18 @@ mysql_ha_fix_cond_and_key(SQL_HANDLER *handler, > Item *item; > key_part_map keypart_map; > uint key_len; > + const KEY *c_key= table->s->key_info + handler->keyno; > + > + if ((c_key->flags & HA_SPATIAL) || > + c_key->algorithm == HA_KEY_ALG_FULLTEXT || > + (ha_rkey_mode != HA_READ_KEY_EXACT && > + (table->file->index_flags(handler->keyno, 0, TRUE) & > + (HA_READ_NEXT | HA_READ_PREV | HA_READ_RANGE)) == 0)) This is strange. Do you mean that SPATIAL and FULLTEXT indexes have HA_READ_NEXT or HA_READ_PREV or HA_READ_RANGE? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MDEV-16094 Crash when using AS OF with a stored function
by Alexander Barkov 15 May '18

15 May '18

Hi Sergei, Please review a patch for MDEV-16094. Thanks!

2 2

Re: [Maria-developers] review of MDEV-16101 ADD PARTITION on table partitioned by list does not work …
by Jacob Mathew 14 May '18

14 May '18

Hi Holyfoot, Your fix creates more lists, all of them containing one element, but it does only one reorganization. My fix does a reorganization whenever a 33rd element is processed but creates less lists. I agree that reducing the number of reorganizations to one is a better way to fix the problem. So I will use your fix. Thanks, Jacob Jacob B. Mathew Spider and Server Developer MariaDB Corporation +1 408 655 8999 (mobile) jacob.b.mathew (Skype) jacob.mathew(a)MariaDB.com On Sun, May 13, 2018 at 2:36 PM, Alexey Botchkov <holyfoot(a)mariadb.com> wrote: > Hi, Jacob. > > Your fix: > https://github.com/MariaDB/server/commit/85bbea8dbb9c01defa993e44a5b16c > 43ef851fbd > > I don't like the way your fix work. > So after we get the 32-th item we do the reorganize_into_single_ > field_col_val() > trick that turns one element with 32 values into 32 elements with a single > value. > Then as we get 33-th element, we again put it into that first element, and > do so for > the next 31 element then call the reorganize_ again. > > I think we should just keep the 'one-value' list after the 'reorganize_' > So my fix would be like this: > --- a/sql/partition_info.cc > +++ b/sql/partition_info.cc > @@ -1996,9 +1996,11 @@ part_column_list_val *partition_info::add_column_value(THD > *thd) > into the structure used for 1 column. After this we call > ourselves recursively which should always succeed. > */ > + num_columns= curr_list_object; > if (!reorganize_into_single_field_col_val(thd)) > { > - DBUG_RETURN(add_column_value(thd)); > + if (!init_column_part(thd)) > + DBUG_RETURN(add_column_value(thd)); > } > > > Best regards. > HF > >

1 0

Re: [Maria-developers] 5651e72: MDEV-15465 Server crash or ASAN heap-use-after-free in Item_func_match::cleanup upon using FT search with partitioning.
by Sergei Golubchik 14 May '18

14 May '18

Hi, Alexey! On May 14, Alexey Botchkov wrote: > Hi Sergei, > > I was wrong - this fix is only needed in 10.3, since the FT keys > are available for PARTITION tables. > So going to push it to the 10.3. Is it ok with you? Yes, thanks! Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] 4861977ce76: MDEV-14695: Assertion `n < m_size' failed in Bounds_checked_array<Element_type>::operator
by Vicențiu Ciorbaru 14 May '18

14 May '18

Hi Varun! I will take a second look at this but a first minor comment is to not save two characters by shortening fields to flds, it makes reading unnatural. Vicentiu On Sat, 12 May 2018 at 21:45 Varun <varunraiko1803(a)gmail.com> wrote: > revision-id: 4861977ce76761fefb9779a774984b6afa818f2e > (mariadb-10.2.5-636-g4861977ce76) > parent(s): 8b26fea83572cf5c043721b7835c3828937f9c27 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-05-13 00:09:15 +0530 > message: > > MDEV-14695: Assertion `n < m_size' failed in > Bounds_checked_array<Element_type>::operator > > In this issue we hit the assert because we are adding addition fields to > the field JOIN::all_fields list. This > is done because HEAP tables can't index BIT fields so we need to use an > additional hidden field for grouping because later it will be > converted to a LONG field. Original field will remain of the BIT type and > will be returned. This happens when we convert DISTINCT to > GROUP BY. > > The solution is to take into account the number of such hidden fields that > would be added to the field JOIN::all_fields list while calculating > the size of the ref_pointer_array. > > --- > sql/sql_base.cc | 9 ++++++--- > sql/sql_base.h | 4 ++-- > sql/sql_delete.cc | 3 ++- > sql/sql_lex.cc | 8 +++++++- > sql/sql_lex.h | 5 +++++ > sql/sql_select.cc | 4 +++- > sql/sql_union.cc | 1 + > 7 files changed, 26 insertions(+), 8 deletions(-) > > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > index 95ff41cb6f4..81ccfe5e514 100644 > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -6919,7 +6919,7 @@ static bool setup_natural_join_row_types(THD *thd, > > int setup_wild(THD *thd, TABLE_LIST *tables, List<Item> &fields, > List<Item> *sum_func_list, > - uint wild_num) > + uint wild_num, uint *hidden_bit_flds) > { > if (!wild_num) > return(0); > @@ -6960,7 +6960,7 @@ int setup_wild(THD *thd, TABLE_LIST *tables, > List<Item> &fields, > else if (insert_fields(thd, ((Item_field*) item)->context, > ((Item_field*) item)->db_name, > ((Item_field*) item)->table_name, &it, > - any_privileges)) > + any_privileges, hidden_bit_flds)) > { > if (arena) > thd->restore_active_arena(arena, &backup); > @@ -7425,7 +7425,7 @@ bool get_key_map_from_key_list(key_map *map, TABLE > *table, > bool > insert_fields(THD *thd, Name_resolution_context *context, const char > *db_name, > const char *table_name, List_iterator<Item> *it, > - bool any_privileges) > + bool any_privileges, uint *hidden_bit_flds) > { > Field_iterator_table_ref field_iterator; > bool found; > @@ -7545,6 +7545,9 @@ insert_fields(THD *thd, Name_resolution_context > *context, const char *db_name, > else > it->after(item); /* Add 'item' to the SELECT list. */ > > + if (item->type() == Item::FIELD_ITEM && item->field_type() == > MYSQL_TYPE_BIT) > + (*hidden_bit_flds)++; > + > #ifndef NO_EMBEDDED_ACCESS_CHECKS > /* > Set privilege information for the fields of newly created views. > diff --git a/sql/sql_base.h b/sql/sql_base.h > index d6063f1b771..8d0dcb2f197 100644 > --- a/sql/sql_base.h > +++ b/sql/sql_base.h > @@ -154,11 +154,11 @@ bool fill_record_n_invoke_before_triggers(THD *thd, > TABLE *table, > enum trg_event_type event); > bool insert_fields(THD *thd, Name_resolution_context *context, > const char *db_name, const char *table_name, > - List_iterator<Item> *it, bool any_privileges); > + List_iterator<Item> *it, bool any_privileges, uint > *hidden_bit_flds); > void make_leaves_list(THD *thd, List<TABLE_LIST> &list, TABLE_LIST > *tables, > bool full_table_list, TABLE_LIST *boundary); > int setup_wild(THD *thd, TABLE_LIST *tables, List<Item> &fields, > - List<Item> *sum_func_list, uint wild_num); > + List<Item> *sum_func_list, uint wild_num, uint * > hidden_bit_flds); > bool setup_fields(THD *thd, Ref_ptr_array ref_pointer_array, > List<Item> &item, enum_mark_columns mark_used_columns, > List<Item> *sum_func_list, List<Item> *pre_fix, > diff --git a/sql/sql_delete.cc b/sql/sql_delete.cc > index 7b7a7e3f804..7bae7736cf3 100644 > --- a/sql/sql_delete.cc > +++ b/sql/sql_delete.cc > @@ -760,7 +760,8 @@ l > select_lex->leaf_tables, FALSE, > DELETE_ACL, SELECT_ACL, TRUE)) > DBUG_RETURN(TRUE); > - if ((wild_num && setup_wild(thd, table_list, field_list, NULL, > wild_num)) || > + if ((wild_num && setup_wild(thd, table_list, field_list, NULL, wild_num, > + &select_lex->hidden_bit_flds)) || > setup_fields(thd, Ref_ptr_array(), > field_list, MARK_COLUMNS_READ, NULL, NULL, 0) || > setup_conds(thd, table_list, select_lex->leaf_tables, conds) || > diff --git a/sql/sql_lex.cc b/sql/sql_lex.cc > index 3e36cac96b9..49150c863fe 100644 > --- a/sql/sql_lex.cc > +++ b/sql/sql_lex.cc > @@ -2143,6 +2143,7 @@ void st_select_lex::init_query() > m_agg_func_used= false; > window_specs.empty(); > window_funcs.empty(); > + hidden_bit_flds= 0; > } > > void st_select_lex::init_select() > @@ -2673,6 +2674,10 @@ ulong st_select_lex::get_table_join_options() > > bool st_select_lex::setup_ref_array(THD *thd, uint order_group_num) > { > + > + if (!((options & SELECT_DISTINCT) && !group_list)) > + hidden_bit_flds= 0; > + > // find_order_in_list() may need some extra space, so multiply by two. > order_group_num*= 2; > > @@ -2687,7 +2692,8 @@ bool st_select_lex::setup_ref_array(THD *thd, uint > order_group_num) > select_n_reserved + > select_n_having_items + > select_n_where_fields + > - order_group_num) * 5; > + order_group_num + > + hidden_bit_flds) * 5; > if (!ref_pointer_array.is_null()) > { > /* > diff --git a/sql/sql_lex.h b/sql/sql_lex.h > index 6d281930d1f..8283496604e 100644 > --- a/sql/sql_lex.h > +++ b/sql/sql_lex.h > @@ -956,6 +956,11 @@ class st_select_lex: public st_select_lex_node > > /* it is for correct printing SELECT options */ > thr_lock_type lock_type; > + /* > + it counts the number of bit fields in the SELECT list. These are used > when DISTINCT is > + converted to a GROUP BY involving BIT fields. > + */ > + uint hidden_bit_flds; > > void init_query(); > void init_select(); > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index 4cd4754596b..83953661ce3 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -799,7 +799,9 @@ JOIN::prepare(TABLE_LIST *tables_init, > select_lex != select_lex->master_unit()->global_parameters()) > real_og_num+= select_lex->order_list.elements; > > - if (setup_wild(thd, tables_list, fields_list, &all_fields, wild_num)) > + DBUG_ASSERT(select_lex->hidden_bit_flds == 0); > + if (setup_wild(thd, tables_list, fields_list, &all_fields, wild_num, > + &select_lex->hidden_bit_flds)) > DBUG_RETURN(-1); > if (select_lex->setup_ref_array(thd, real_og_num)) > DBUG_RETURN(-1); > diff --git a/sql/sql_union.cc b/sql/sql_union.cc > index 3fe8a24f8bd..42904e341b6 100644 > --- a/sql/sql_union.cc > +++ b/sql/sql_union.cc > @@ -1474,6 +1474,7 @@ bool st_select_lex::cleanup() > } > inner_refs_list.empty(); > exclude_from_table_unique_test= FALSE; > + hidden_bit_flds= 0; > DBUG_RETURN(error); > } > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] MDEV-6111: Optimizer trace: features and milestones
by Sergey Petrunia 11 May '18

11 May '18

Hi Zhzhzoo, Please find below some thoughts about features/milestones in the project. Items at the top of the list should be done first. The list is not necesarrily complete, any comments/additions are welcome. - Optimizer trace collection API (is it different from Json_writer that we have? Does MySQL use the same API to produce EXPLAIN FORMAT=JSON and produce optimizer_trace? Should we follow them?) - Basic optimizer trace collection (just print the query and "hello":"world" instead of trace). Clear the trace data at query start, collect the trace if tracing is switched on (and do not collect if it is off) - INFORMATION_SCHEMA.OPTIMIZER_TRACE table in I_S database. Produce optimizer_trace content for features common with MySQL: = condition rewrite = partition pruning = range analysis = ref/eq_ref analysis (update_ref_and_keys) = (TODO what else?) Produce optimizer_trace content for MariaDB-specific features: = merged semi-join conversion = non-merged semi-join conversion = join condition pushdown = (TODO what else?) - optimizer_trace_limit/optimizer_trace_offset (without this feature, just save one last trace) - Ability to produce either a pretty-printed or one-line optimizer trace - Security = INFORMATION_SCHEMA.OPTIMIZER_TRACE.INSUFFICIENT_PRIVILEGES = Who can view optimizer_trace? = Figure out and implement the necessary security restrictions - Memory limits = Honor the @@optimizer_trace_max_mem_size setting. BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

2 1

[Maria-developers] Please review MDEV-16134 Wrong I_S.COLUMNS.CHARACTER_XXX_LENGTH value for compressed columns
by Alexander Barkov 10 May '18

10 May '18

Hi Sergey, Please review a patch for MDEV-16134. Thanks!

1 0

[Maria-developers] MDEV-15935: Connection Redirection Mechanism in MariaDB Client/Server Protocol
by xiangyu hu 09 May '18

09 May '18

Hi developers, There is an ongoing discussion regarding to the proposal of connection redirection in JIRA. Please kindly take a look if you aren't aware of it. Link is: https://jira.mariadb.org/browse/MDEV-15935. Thank you, Xiangyu

1 0

[Maria-developers] MDEV-15450: Add Host Name Field to MariaDB Client Handshake Protocol
by xiangyu hu 09 May '18

09 May '18

Hello devs, We have initiated a proposal in JIRA where we described the necessity of adding host_name into login packet attributes. I would like to broaden the audience here to bring up more discussions. Please take a look at MDEV-15450 (https://jira.mariadb.org/browse/MDEV-15450) [MDEV-15450] Add Host Name Field to MariaDB Client ...<https://jira.mariadb.org/browse/MDEV-15450> jira.mariadb.org Why. In original, one MariaDB server instance is identified by one address (IP + port). That is not enough for a shared proxy for multi servers. Thanks, Xiangyu

1 0

Re: [Maria-developers] eb3e9cd: MDEV-15813 ASAN use-after-poison in hp_hashnr upon HANDLER READ on a versioned HEAP table.
by Sergei Golubchik 09 May '18

09 May '18

Hi, Alexey! On May 08, Alexey Botchkov wrote: > revision-id: eb3e9cdbe86b5a68c3ec8a9f26165259e4ca7eec (mariadb-10.3.6-125-geb3e9cd) > parent(s): bd09c5ca866e273b6cebbd9a15c51c82bfa3ac9b > committer: Alexey Botchkov > timestamp: 2018-05-08 14:19:55 +0400 > message: > > MDEV-15813 ASAN use-after-poison in hp_hashnr upon HANDLER READ on a versioned HEAP table. > > Returns an error if we try to use a key in a query it declared unable to > handle. > > diff --git a/sql/share/errmsg-utf8.txt b/sql/share/errmsg-utf8.txt > index c55ac4f..7446a60 100644 > --- a/sql/share/errmsg-utf8.txt > +++ b/sql/share/errmsg-utf8.txt > @@ -7915,3 +7915,5 @@ ER_UPDATED_COLUMN_ONLY_ONCE > eng "The column %`s.%`s cannot be changed more than once in a single UPDATE statement" > ER_EMPTY_ROW_IN_TVC > eng "Row with no elements is not allowed in table value constructor in this context" > +ER_KEY_DOESNT_SUPPORT > + eng "%s index %`s does not support this operation" I think this can be safely pushed into 10.2. But not earlier, we can only add new error messages in the latest GA version, not in previos GA's. > diff --git a/sql/sql_handler.cc b/sql/sql_handler.cc > index 09883d8..12e4b5a 100644 > --- a/sql/sql_handler.cc > +++ b/sql/sql_handler.cc > @@ -661,12 +661,30 @@ mysql_ha_fix_cond_and_key(SQL_HANDLER *handler, > key_part_map keypart_map; > uint key_len; > > + if (ha_rkey_mode != HA_READ_KEY_EXACT && > + (table->file->index_flags(handler->keyno, 0, TRUE) & > + (HA_READ_NEXT | HA_READ_PREV | HA_READ_RANGE)) == 0) I suspect there can be hypothetical cases where an index supports some specific ha_rkey_mode values and has some specific flags, but not all of them, like you test here. But as the only real use case is HASH vs BTREE now, your check is fine. No need overengineer a complex test that we cannot possibly test now. > + { > + my_error(ER_KEY_DOESNT_SUPPORT, MYF(0), > + table->file->index_type(handler->keyno), keyinfo->name); > + return 1; > + } > + > if (key_expr->elements > keyinfo->user_defined_key_parts) > { > my_error(ER_TOO_MANY_KEY_PARTS, MYF(0), > keyinfo->user_defined_key_parts); > return 1; > } > + else if (key_expr->elements < keyinfo->user_defined_key_parts && > + (table->file->index_flags(handler->keyno, 0, TRUE) & > + HA_ONLY_WHOLE_INDEX)) > + { > + my_error(ER_KEY_DOESNT_SUPPORT, MYF(0), > + table->file->index_type(handler->keyno), keyinfo->name); > + return 1; Good point! > + } This is ok to push. Just two questions: 1. Why did you add it in sql_handler.cc and not in the class handler method, like ha_index_read_map or ha_index_next ? 2. Could you add tests for fulltext/rtree indexes too, please? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] GSoC 2018 - MDEV-12313 - 30th of April -> 6th of May Update
by Teodor-Vicentiu Niculescu 08 May '18

08 May '18

Greetings, First of all, I want to thank you for accepting me for this years GSoC. I plan to do my best during the following months in order to implement the SQL Statement specified in the proposal. https://docs.google.com/document/d/1dA7YlDPlRG43LhfGlHXnHJiERfjExLG9qS7BqXh… In the last week (30th of April to 6th of May) I have started the first task of the proposal, specifically extending the grammar. After receiving some initial hints from Mr. Ciorbaru, concerning where to start looking in the files ( ANALYZE_SYM in sql/sql_yacc.yy ), I quickly discovered I have no knowledge of GNU Bison. I took my time to look through the GNU Bison online documentation, I have compiled tests and written code in order to better understand parsing. Now, sql_yacc.yy is starting to make sense, which is why I plan to finish the parsing part of the project by Sunday (13th of May). Until 15th of June my schedule will be quite busy because finals are approaching. You can still contact me by e-mail. Advice and constructive criticism is always much appreciated. Thank you for this amazing opportunity. Sincerely, Teodor

1 0

Re: [Maria-developers] fa0b71b0e19: MDEV-10259 mysqld crash with certain statement length and...
by Sergei Golubchik 08 May '18

08 May '18

Hi, Sachin! On May 08, Sachin Setiya wrote: > diff --git a/unittest/sql/mf_iocache-t.cc b/unittest/sql/mf_iocache-t.cc > index 31f98562521..d5e2ffb095e 100644 > --- a/unittest/sql/mf_iocache-t.cc > +++ b/unittest/sql/mf_iocache-t.cc > @@ -187,10 +187,71 @@ void mdev9044() > close_cached_file(&info); > } > > +/* 2 Reads in cache makes second read to fail (only if first read uses temp file) */ > +void mdev10259() > +{ > + int res; > + uchar buf[CACHE_SIZE + 200]; > + memset(buf, FILL, sizeof(buf)); > + > + diag("MDEV-10259- mysqld crash with certain statement length and order with" > + " Galera and encrypt-tmp-files=1"); > + > + init_io_cache_encryption(); > + > + res= open_cached_file(&info, 0, 0, CACHE_SIZE, 0); > + ok(res == 0, "open_cached_file" INFO_TAIL); > + > + res= my_b_write(&info, buf, sizeof(buf)); > + ok(res == 0 && info.pos_in_file == CACHE_SIZE, "large write" INFO_TAIL); > + > + res= my_b_flush_io_cache(&info, 1); > + ok(res == 0, "flush" INFO_TAIL); I think this misses the point. The bug happened when the data is in the cache, and there's no file to read from, so when the data is lost - it's forever. But my_b_flush_io_cache() writes the buffer to a file, so my_b_fill() can read it even if the buffer is emptied. Meaning, I suspect that if you remove your fix from mf_iocache_encr.cc, this unit test will still pass. Please double-check that your test fails without a fix. Ok to push after that. > + res= reinit_io_cache(&info, READ_CACHE, 0, 0, 0); > + ok(res == 0, "reinit READ_CACHE" INFO_TAIL); Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 7107565: MDEV-10259 mysqld crash with certain statement length and...
by Sergei Golubchik 08 May '18

08 May '18

Hi, Sachin! Great work! On Mar 27, Sachin Setiya wrote: > revision-id: 710756527ec8589bce33bc834a1d8e6604266b6d (mariadb-10.1.31-47-g7107565) > parent(s): 8152c52e1af608da5d288a4e6fc82e2d804e64cd > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2018-03-27 04:24:57 +0530 > message: > > MDEV-10259 mysqld crash with certain statement length and... > order with Galera and encrypt-tmp-files=1 > > Problem:- If trans_cache (IO_CACHE) uses encrypted tmp file > then on next DML server will crash. > > Case:- > Lets take a case , we have a table t1 , We try to do 2 inserts in t1 > 1. A really long insert so that trans_cache has to use temp_file > 2. Just a small insert > > Analysis:- Actually server crashes from inside of galera > library. > /lib64/libc.so.6(abort+0x175)[0x7fb5ba779dc5] > /usr/lib64/galera/libgalera_smm.so(_ZN6galera3FSMINS_9TrxHandle5State... > mysys/stacktrace.c:247(my_print_stacktrace)[0x7fb5a714940e] > sql/signal_handler.cc:160(handle_fatal_signal)[0x7fb5a715c1bd] > sql/wsrep_hton.cc:257(wsrep_rollback)[0x7fb5bcce923a] > sql/wsrep_hton.cc:268(wsrep_rollback)[0x7fb5bcce9368] > sql/handler.cc:1658(ha_rollback_trans(THD*, bool))[0x7fb5bcd4f41a] > sql/handler.cc:1483(ha_commit_trans(THD*, bool))[0x7fb5bcd4f804] > > but actual issue is not in galera but in mariadb, because for 2nd > insert we should never call rollback. We are calling rollback because > log_and_order fails it fails because write_cache fails , It fails > because after reinit_io_cache(trans_cache) , my_b_tell says 0 so we > look into tmp_file for data , which is obviously wrong since temp > was used for previous insert and it no longer exist. Add here something like: wsrep_write_cache_inc() reads the IO_CACHE in a loop, filling it with my_b_fill() until it returns "0 bytes read". Later MYSQL_BIN_LOG::write_cache() does the same. wsrep_write_cache_inc() assumes that reading a zero bytes past EOF leaves the old data in the cache. Let's do that. > diff --git a/sql/mf_iocache_encr.cc b/sql/mf_iocache_encr.cc > index 149e6fe..41e1333 100644 > --- a/sql/mf_iocache_encr.cc > +++ b/sql/mf_iocache_encr.cc > @@ -49,8 +49,7 @@ static int my_b_encr_read(IO_CACHE *info, uchar *Buffer, size_t Count) > > if (pos_in_file == info->end_of_file) > { here, a comment: /* reading past EOF should not empty the cache */ > - info->read_pos= info->read_end= info->buffer; > - info->pos_in_file= pos_in_file; > + info->read_pos= info->read_end; > info->error= 0; > DBUG_RETURN(MY_TEST(Count)); > } and add a test for it into unittest/sql/mf_iocache-t.cc. Something simple, like, write into a cache, reinit to READ, fill, fill, reinit to READ, verify the data is still there. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 2

Re: [Maria-developers] bea915f: MDEV-15965 Invisible columns and LOAD DATA don't work well together:...
by Sergei Golubchik 07 May '18

07 May '18

Hi, sachin! On Apr 26, sachin wrote: > revision-id: bea915f21a2a2771ce7f9e19dcb9d6893f33f7bd (mariadb-10.3.6-47-gbea915f) > parent(s): 326db1a2aaa9b275a1a21a863e8cd2d9fa1b1d5f > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2018-04-26 16:49:27 +0530 > message: > > MDEV-15965 Invisible columns and LOAD DATA don't work well together:... > ER_WARN_TOO_FEW_RECORDS > > Fix mysql_load iterator to skip invisible fields. > > diff --git a/sql/sql_load.cc b/sql/sql_load.cc > index cfa92f1..fbb3926 100644 > --- a/sql/sql_load.cc > +++ b/sql/sql_load.cc > @@ -444,6 +444,8 @@ int mysql_load(THD *thd, const sql_exchange *ex, TABLE_LIST *table_list, > field_iterator.set(table_list); > for (; !field_iterator.end_of_fields(); field_iterator.next()) > { > + if (field_iterator.field()->invisible > VISIBLE) > + continue; Oh, that's all? And the syntax with SET just worked automatically? Cool. ok to push! Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 326db1a: Mdev-14853 Grant does not work correctly when table contains...
by Sergei Golubchik 07 May '18

07 May '18

Hi, Sachin! On Apr 26, sachin wrote: > revision-id: 326db1a2aaa9b275a1a21a863e8cd2d9fa1b1d5f (mariadb-10.3.6-46-g326db1a) > parent(s): 9477a2a9ba17c0db362e2bb39d5048e369096f39 > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2018-04-26 12:47:25 +0530 > message: > > Mdev-14853 Grant does not work correctly when table contains... > SYSTEM_INVISIBLE or COMPLETELY_INVISIBLE > > This commit does multiple things to solve this mdev > 1st add field into the parameter of check_column_grant_in_table_ref, so that > we can find out field invisibility. > 2nd If field->invisible >= INVISIBLE_SYSTEM skip access check and simple > grant access. Looks ok. A couple of comments about the test: 1. I suppose now you can use system versioning to test system invisible fields, so there's no need to limit this test to debug-only builds. 2. please test a case when a user has *no* privileges on the table at all. To make sure he cannot select always-readable system invisible fields. and change the capitalization in the commit comment to say MDEV not Mdev - I suspect this was the reason why Jira didn't pick up your commit. Thanks! Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 5651e72: MDEV-15465 Server crash or ASAN heap-use-after-free in Item_func_match::cleanup upon using FT search with partitioning.
by Sergei Golubchik 07 May '18

07 May '18

Hi, Alexey! On May 07, Alexey Botchkov wrote: > Yes, it is an issue in 5.5. > But we don't always fix bugs in the earliest version possible. > > Ok. Do i understand it right that the patch is ok to push into 5.5? Yes. > Should i then propagate the patch upt to 10.3 muself? No, don't bother. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 5651e72: MDEV-15465 Server crash or ASAN heap-use-after-free in Item_func_match::cleanup upon using FT search with partitioning.
by Sergei Golubchik 06 May '18

06 May '18

Hi, Alexey! On May 03, Alexey Botchkov wrote: > revision-id: 5651e72b8dd20e29a4963723773ec6bcfac457ba (mariadb-10.3.6-102-g5651e72) > parent(s): 73a10cbcc5178ae5378abb821428d35d3276b4da > committer: Alexey Botchkov > timestamp: 2018-05-03 12:42:56 +0400 > message: > > MDEV-15465 Server crash or ASAN heap-use-after-free in Item_func_match::cleanup upon using FT search with partitioning. > > The Item_func_match::cleanup() uses table's internals so it's not safe > to call it after the close_thread_tables(). Let's call it in > st_select_lex::cleanup(). Oops. Agree, thanks. But why do you fix it in 10.3? Looks like a 5.5+ bug. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] cc0bc69: MDEV-15813 ASAN use-after-poison in hp_hashnr upon HANDLER READ on a
by Sergei Golubchik 06 May '18

06 May '18

Hi, Alexey! On May 06, Alexey Botchkov wrote: > revision-id: cc0bc69f2d28aace12487f930677abcdffa5ce97 (mariadb-10.3.6-102-gcc0bc69) > parent(s): 73a10cbcc5178ae5378abb821428d35d3276b4da > committer: Alexey Botchkov > timestamp: 2018-05-06 02:37:20 +0400 > message: > > MDEV-15813 ASAN use-after-poison in hp_hashnr upon HANDLER READ on a > versioned HEAP table. > > hp_rkey now returns an error if we try to use HASH key where it doesn't > work. > > diff --git a/sql/share/errmsg-utf8.txt b/sql/share/errmsg-utf8.txt > index c55ac4f..fcd20d8 100644 > --- a/sql/share/errmsg-utf8.txt > +++ b/sql/share/errmsg-utf8.txt > @@ -7915,3 +7915,5 @@ ER_UPDATED_COLUMN_ONLY_ONCE > eng "The column %`s.%`s cannot be changed more than once in a single UPDATE statement" > ER_EMPTY_ROW_IN_TVC > eng "Row with no elements is not allowed in table value constructor in this context" > +ER_KEY_DOESNT_SUPPORT > + eng "HASH index %`s does not support this operation" Use "%s index %`s does not support this operation" please. First parameter could be taken from handler::index_type(). By the way, check what happens when the index is a fulltext or rtree. Is it also a crash or this case is handled somehow? If it does, the same check should take care of HASH indexes, no need to do a separate solution for them. > diff --git a/storage/heap/hp_rkey.c b/storage/heap/hp_rkey.c > index 692e1a0..a7b81ae 100644 > --- a/storage/heap/hp_rkey.c > +++ b/storage/heap/hp_rkey.c > @@ -60,6 +60,14 @@ int heap_rkey(HP_INFO *info, uchar *record, int inx, const uchar *key, > } > else > { > + /* > + HASH key doesn't support search other than equality, > + and doesn't support prefix search. > + */ > + if (find_flag != HA_READ_KEY_EXACT || > + (1UL << share->keydef[inx].keysegs) != keypart_map + 1) > + DBUG_RETURN(my_errno= HA_ERR_KEY_DOESNT_SUPPORT); > + No, please, not here. The parent handler object knows index types, it knows index flags and it can return an error if the corresponding index flag is missing. No need to push this check down to the engine if the server can do it in, say, ha_index_read. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] MDEV-15813 ... HANDLER READ on a versioned HEAP table
by Sergei Golubchik 05 May '18

05 May '18

Hi, Alexey! On May 04, Alexey Botchkov wrote: > Hi, Sergei, guys. > > This bug iis more like a feature of the HASH indexes. > And i came upon other 'features' while experimenting. > > Firstly the 'range' queries like > HANDLER h READ `PRIMARY` < (5); > cannot be naturally handled by the HASH keys. Only equality queries > ...'PRIMARY' = (5); make sence. > And these 'range' queries just return wrong empty result. > I'd say we need to fire an error in this case. Do you agree? Yes. Like "HASH index `PRIMARY` does not support this operation" ? > Second problem - the SYSTEM VERSIONING. > It adds an extra column to the key, which makes this key virtually > useless. I think for the VERSIONING we should either replace the HASH > key with the RBTREE, or get the row_end data from the table after > getting records from the HASH. > What is your idea how to deal with this? row_end must be a part of every unique key, because uniqueness of current data should not be affected by historical, deleted, rows. In this particular use case, HANDLER is a low level command, it doesn't see high-level features like versioning. So, for HANDLER row_end is just a normal key part, should be used as HANDLER h READ `PRIMARY` = (5, '2038-01-19 03:14:07.999999'); Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] : GSoC Project Optimizer Trace
by Sergey Petrunia 04 May '18

04 May '18

Hi Zhzhzoo, Please find below some thoughts about the optimizer trace project * The MDEV for the optimizer trace task is https://jira.mariadb.org/browse/MDEV-6111. Let's keep it up to date with all the changes. * I think we should target MySQL-like functionality of optimizer_trace. At least, for the first milestone. We may decide to make some things better but working in a similar way to MySQL is a nice first goal. * MariaDB's optimizer trace will contain JSON and produce similar content to MySQL. However, MariaDB's optimizer is different, so one cannot expect that the JSON document will have exactly the same fields/objects/etc. (MariaDB' EXPLAIN FORMAT=JSON output doesn't match the one in MySQL, either. And in MySQL, the output varies across its versions.) * MySQL prints conditions with heavy over-quoting. Example: "original_condition": "((`A`.`a` = `B`.`a`) and ((`A`.`a` + `B`.`a`) < 10))", MariaDB has the code to print the condition without redundant quotes (see how EXPLAIN FORMAT=JSON does it). * I assume MySQL has some class for printing well-formed JSON documents. MariaDB also has one. It is in sql/my_json_writer.h, see class Json_writer. It handles printing into JSON, and also does pretty-printing. When I looked at the code last time, it did pretty-printing better than MySQL's code for EXPLAIN FORMAT=JSON did. It would be nice to use Json_writer for collecting optimizer trace output, too. (there is a caveat: Json_writer was written specifically for producing EXPLAIN FORMAT=JSON output so it may lack some features. You may need to extend it). BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] MariaDB Developers Unconference in June
by Ian Gilfillan 03 May '18

03 May '18

Hi A reminder about the next MariaDB Developers Unconference, which will be held in Tampera, Finland, from 26 June to 29 June. This will be a four-day event; the first day focusing on new contributors, the 2nd and 3rd the regular Unconference sessions and discussions, and the 4th day on getting patches reviewed and implemented, taking advantage of the chance to have in-person discussions. Details, as well as where to register and how to suggest sessions, are at https://mariadb.org/2018-2-developers-unconference-in-finland/

1 0

Re: [Maria-developers] [External] Re: Questions regarding closing partitions in MDEV-11084
by Mattias Jonsson 30 Apr '18

30 Apr '18

Hi Holyfoot, Sergei and Jacob, Thank you for looking into this. Jacob there are no Jira ticket (that I am aware of) also I do see it as two different problems with the same source. 1) Spider engine does not handle opening/closing specific partitions (as I understand it currently relies on opening/closing all partitions at once in order). Currently this is a crashing bug! 2) MDEV-11084 makes explicit partition selection work like FLUSH TABLE, due to the closing partitions part. I think it would be nice if you could create tickets for those two issues (or at least for the first one). Regards Mattias On Thu, Apr 26, 2018 at 11:26 AM, Alexey Botchkov <holyfoot(a)mariadb.com> wrote: > No, there's no JIRA task for that problem. > So i guess you can create one :) > > Best regards. > HF > > On Thu, Apr 26, 2018 at 4:50 AM, Jacob Mathew <jacob.mathew(a)mariadb.com> > wrote: > >> Hi Mattias and Holyfoot, >> >> Is there a Jira bug for this problem? If not I can create a bug for it >> and assign it to Holyfoot. >> >> Thanks, >> Jacob >> >> Jacob B. Mathew >> Spider and Server Developer >> MariaDB Corporation >> +1 408 655 8999 (mobile) >> jacob.b.mathew (Skype) >> jacob.mathew(a)MariaDB.com >> >> >> On Tue, Apr 17, 2018 at 9:11 AM, Mattias Jonsson < >> mattias.jonsson(a)booking.com> wrote: >> >>> Hi Holyfoot, >>> >>> On Mon, Apr 16, 2018 at 3:38 PM, Alexey Botchkov <holyfoot(a)askmonty.org> >>> wrote: >>> > Hi, Mattias, guys! >>> > >>> > While investigating the crash, i'd like to discuss that >>> > >>> >> it seems to close partitions whenever it >>> >> is not used in a statement (i.e. require it to be reopened in the next >>> >> statement that would use another partition >>> > >>> > >>> > Yes, it does that, handling statements with the specified 'PARTITION' >>> > option. >>> > The patch supposed to solve the problem when there are too many >>> partitions >>> > opened, >>> > so i think it must close the unused partitions sometime. >>> > No, it doesn't have to happen that often. I planned to check the >>> > table_open_cache >>> > variable before the forced close. But decided not to do that initially >>> - as >>> > it simplified testing, >>> > and i thought if someone uses the PARTITION option, he would stick to >>> using >>> > this partition >>> > anyway. And i forgot about that issue. >>> >>> The reason for not closing partitions in this case is that it turns >>> the 'SELECT col FROM t PARTITION(p)' almost into a 'FLUSH TABLES t' >>> which is kind of unexpected. >>> I did a test mixing PK selects with and without 'PARTITION (p)' clause >>> and that shows it will close all but 1 partition and then (re)open all >>> but 1 partition. >>> >>> Think of the case when a server runs in production with a heavily >>> partitioned table serving simple PK queries and then someone runs a >>> query with explicit PARTITION selection, then it will introduce a >>> short stall. First for the query itself (closing all but 1 partitions) >>> and then for the next simple PK query using the same table (opening >>> all but 1 partition). >>> >>> As I read the bug report: the reporter wants to avoid opening all >>> partitions. Not that it keeps the partitions open in the table open >>> cache (which is an issue on the architectural level of partitioning >>> not really fitting into the open table cache). >>> >>> I attached a diff with the test and results (I also added handler >>> status variables to show my point). The diff is against >>> b4a2baffa82e5c07b96a1c752228560dcac1359b. >>> >>> Here is the part of the result file that shows what I mean with extra >>> comments prepended by MJ>: >>> CREATE TABLE t1 (a int PRIMARY KEY) >>> ENGINE = InnoDB >>> PARTITION BY HASH (a) PARTITIONS 1000; >>> INSERT INTO t1 VALUES (0), (1), (2), (3); >>> FLUSH STATUS; >>> >>> SELECT a FROM t1 PARTITION(p0) WHERE a = 0; >>> a >>> 0 >>> SHOW SESSION STATUS >>> WHERE VARIABLE_NAME LIKE 'HANDLER_%' AND VALUE > 0; >>> Variable_name Value >>> MJ> Here it closes all but one partitions. >>> Handler_close 999 >>> >>> SELECT a FROM t1 WHERE a = 0; >>> a >>> 0 >>> SHOW SESSION STATUS >>> WHERE VARIABLE_NAME LIKE 'HANDLER_%' AND VALUE > 0; >>> Variable_name Value >>> Handler_close 1000 >>> Handler_external_lock 2006 >>> MJ> and here it re-opens those 'all but one' partitions. >>> Handler_open 999 >>> >>> >>> Regards >>> Mattias >>> > >>> > >>> > Best regards. >>> > HF >>> > >>> > >>> > >>> > >>> > 13.04.2018 19:07, Mattias Jonsson wrote: >>> >> >>> >> Hi MariaDB Devs, >>> >> >>> >> I tried to evaluate spider engine and found an issue where it crashes, >>> >> most likely due to MDEV-11084 (Stacktrace and reproducible test case >>> >> attached). >>> >> >>> >> That also leads me to wonder about the performance for partitioned >>> >> tables after MDEV-11084, when it seems to close partitions whenever it >>> >> is not used in a statement (i.e. require it to be reopened in the next >>> >> statement that would use another partition, effectively not using the >>> >> open table cache). >>> >> >>> >> I cannot see anything mentioned in the final commit message hinting on >>> >> that it closes partitions not used in the current query, but in the >>> >> previous patches it was mentioned without any reason. >>> >> >>> >> Why does it close the already opened partitions? >>> >> https://github.com/MariaDB/server/blob/10.3/sql/ha_partition.cc#L8365 >>> >> >>> >> I would not mind opening the partitions only when they are to be used >>> >> (even though there are engines that need to be tested more), but >>> >> closing them makes no sense to me performance wise. Also notice that >>> >> the partitions first will be put back into the open table cache and >>> >> then on the next query the non-used partitions will be closed and the >>> >> needed ones be (re)-opened. >>> >> >>> >> Regards >>> >> Mattias Jonsson >>> > >>> > >>> > >>> > >>> >> >> > -- Mattias Jonsson IT Team Lead - Development Booking.com B.V. Herengracht 597 Amsterdam 1017 CE Netherlands Direct +31207125646 [image: Booking.com] <http://www.booking.com/> The world's #1 accommodation site 43 languages, 198+ offices worldwide, 120,000+ global destinations, 1,550,000+ room nights booked every day No booking fees, best price always guaranteed Subsidiary of Booking Holdings Inc. (NASDAQ: BKNG)

3 4

Re: [Maria-developers] [Commits] 326db1a: Mdev-14853 Grant does not work correctly when table contains...
by Vicențiu Ciorbaru 26 Apr '18

26 Apr '18

Hi Sachin! Did you run this test on embedded server? Usually grant related tests require some form of not_embedded.inc include. (check roles suite for examples) Vicentiu On Thu, 26 Apr 2018 at 00:20 sachin <sachin.setiya(a)mariadb.com> wrote: > revision-id: 326db1a2aaa9b275a1a21a863e8cd2d9fa1b1d5f > (mariadb-10.3.6-46-g326db1a) > parent(s): 9477a2a9ba17c0db362e2bb39d5048e369096f39 > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2018-04-26 12:47:25 +0530 > message: > > Mdev-14853 Grant does not work correctly when table contains... > SYSTEM_INVISIBLE or COMPLETELY_INVISIBLE > > This commit does multiple things to solve this mdev > 1st add field into the parameter of check_column_grant_in_table_ref, so > that > we can find out field invisibility. > 2nd If field->invisible >= INVISIBLE_SYSTEM skip access check and simple > grant access. > > --- > mysql-test/main/invisible_field_grant.result | 111 > +++++++++++++++++++++++++++ > mysql-test/main/invisible_field_grant.test | 77 +++++++++++++++++++ > sql/sp_rcontext.cc | 9 ++- > sql/sql_acl.cc | 11 ++- > sql/sql_acl.h | 2 +- > sql/sql_base.cc | 4 +- > 6 files changed, 206 insertions(+), 8 deletions(-) > > diff --git a/mysql-test/main/invisible_field_grant.result > b/mysql-test/main/invisible_field_grant.result > new file mode 100644 > index 0000000..c3ccbb1 > --- /dev/null > +++ b/mysql-test/main/invisible_field_grant.result > @@ -0,0 +1,111 @@ > +set @old_debug= @@debug_dbug; > +create user user_1; > +show grants for user_1; > +Grants for user_1@% > +GRANT USAGE ON *.* TO 'user_1'@'%' > +# create user > +create database d; > +use d; > + > +#System_Invisible > +set debug_dbug= "+d,test_pseudo_invisible"; > +create table t1(a int); > +set debug_dbug=@old_debug; > +insert into t1 values(1); > +select a,invisible from t1; > +a invisible > +1 9 > +grant insert(a) on t1 to user_1; > +grant update(a) on t1 to user_1; > +grant select(a) on t1 to user_1; > +grant delete on t1 to user_1; > +connect con1, localhost, user_1,,test; > +connection con1; > +select user(); > +user() > +user_1@localhost > +use d; > +select * from t1; > +a > +1 > +insert into t1 values(2); > +select * from t1; > +a > +1 > +2 > +insert into t1(a) values(3); > +select * from t1; > +a > +1 > +2 > +3 > +select invisible,a from t1; > +invisible a > +9 1 > +9 2 > +9 3 > +delete from t1 where a =1; > +update t1 set a=1 where a=3; > +select * from t1; > +a > +2 > +1 > +disconnect con1; > + > +#Cleanup > +connection default; > +drop table t1; > +REVOKE ALL PRIVILEGES, GRANT OPTION FROM user_1; > + > +#Completely Invisible > +set debug_dbug= "+d,test_completely_invisible"; > +create table t1(a int); > +insert into t1 values(1); > +select a,invisible from t1; > +a invisible > +1 9 > +set debug_dbug=@old_debug; > +grant insert(a) on t1 to user_1; > +grant update(a) on t1 to user_1; > +grant select(a) on t1 to user_1; > +grant delete on t1 to user_1; > +connect con1, localhost, user_1,,test; > +connection con1; > +select user(); > +user() > +user_1@localhost > +use d; > +select * from t1; > +a > +1 > +insert into t1 values(2); > +select * from t1; > +a > +1 > +2 > +insert into t1(a) values(3); > +select * from t1; > +a > +1 > +2 > +3 > +select invisible,a from t1; > +ERROR 42S22: Unknown column 'invisible' in 'field list' > +delete from t1 where a =1; > +update t1 set a=1 where a=3; > +select * from t1; > +a > +2 > +1 > +disconnect con1; > + > +#Final Cleanup > +connection default; > +set debug_dbug= "+d,test_completely_invisible"; > +select a,invisible from t1; > +a invisible > +2 9 > +1 9 > +drop user user_1; > +drop database d; > +set @old_debug= @@debug_dbug; > diff --git a/mysql-test/main/invisible_field_grant.test > b/mysql-test/main/invisible_field_grant.test > new file mode 100644 > index 0000000..0d627e5 > --- /dev/null > +++ b/mysql-test/main/invisible_field_grant.test > @@ -0,0 +1,77 @@ > +--source include/have_debug.inc > +##TEST for invisible coloumn level 2 > +set @old_debug= @@debug_dbug; > +create user user_1; > +show grants for user_1; > +--echo # create user > +create database d; > +use d; > + > +--echo > +--echo #System_Invisible > +set debug_dbug= "+d,test_pseudo_invisible"; > +create table t1(a int); > +set debug_dbug=@old_debug; > +insert into t1 values(1); > +select a,invisible from t1; > +grant insert(a) on t1 to user_1; > +grant update(a) on t1 to user_1; > +grant select(a) on t1 to user_1; > +grant delete on t1 to user_1; > +connect (con1, localhost, user_1,,test); > +connection con1; > +select user(); > +use d; > +select * from t1; > +insert into t1 values(2); > +select * from t1; > +insert into t1(a) values(3); > +select * from t1; > +select invisible,a from t1; > +delete from t1 where a =1; > +update t1 set a=1 where a=3; > +select * from t1; > +disconnect con1; > +--source include/wait_until_disconnected.inc > + > +--echo > +--echo #Cleanup > +--connection default > +drop table t1; > +REVOKE ALL PRIVILEGES, GRANT OPTION FROM user_1; > + > +--echo > +--echo #Completely Invisible > +set debug_dbug= "+d,test_completely_invisible"; > +create table t1(a int); > +insert into t1 values(1); > +select a,invisible from t1; > +set debug_dbug=@old_debug; > +grant insert(a) on t1 to user_1; > +grant update(a) on t1 to user_1; > +grant select(a) on t1 to user_1; > +grant delete on t1 to user_1; > +connect (con1, localhost, user_1,,test); > +connection con1; > +select user(); > +use d; > +select * from t1; > +insert into t1 values(2); > +select * from t1; > +insert into t1(a) values(3); > +select * from t1; > +--error ER_BAD_FIELD_ERROR > +select invisible,a from t1; > +delete from t1 where a =1; > +update t1 set a=1 where a=3; > +select * from t1; > +disconnect con1; > +--source include/wait_until_disconnected.inc > +--echo > +--echo #Final Cleanup > +connection default; > +set debug_dbug= "+d,test_completely_invisible"; > +select a,invisible from t1; > +drop user user_1; > +drop database d; > +set @old_debug= @@debug_dbug; > diff --git a/sql/sp_rcontext.cc b/sql/sp_rcontext.cc > index 2e9ae23..dc103fa 100644 > --- a/sql/sp_rcontext.cc > +++ b/sql/sp_rcontext.cc > @@ -196,11 +196,12 @@ bool sp_rcontext::init_var_table(THD *thd, > */ > static inline bool > check_column_grant_for_type_ref(THD *thd, TABLE_LIST *table_list, > - const char *str, size_t length) > + const char *str, size_t length, > + Field *fld) > { > #ifndef NO_EMBEDDED_ACCESS_CHECKS > table_list->table->grant.want_privilege= SELECT_ACL; > - return check_column_grant_in_table_ref(thd, table_list, str, length); > + return check_column_grant_in_table_ref(thd, table_list, str, length, > fld); > #else > return false; > #endif > @@ -238,7 +239,7 @@ bool Qualified_column_ident::resolve_type_ref(THD > *thd, Column_definition *def) > { > if (!(rc= check_column_grant_for_type_ref(thd, table_list, > m_column.str, > - m_column.length))) > + m_column.length, src))) > { > *def= Column_definition(thd, src, NULL/*No defaults,no > constraints*/); > def->flags&= (uint) ~NOT_NULL_FLAG; > @@ -302,7 +303,7 @@ bool Table_ident::resolve_table_rowtype_ref(THD *thd, > LEX_CSTRING tmp= src[0]->field_name; > Spvar_definition *def; > if ((rc= check_column_grant_for_type_ref(thd, table_list, > - tmp.str, tmp.length)) || > + tmp.str, > tmp.length,src[0])) || > (rc= !(src[0]->field_name.str= thd->strmake(tmp.str, > tmp.length))) || > (rc= !(def= new (thd->mem_root) Spvar_definition(thd, *src)))) > break; > diff --git a/sql/sql_acl.cc b/sql/sql_acl.cc > index de4e201..1c154a1 100644 > --- a/sql/sql_acl.cc > +++ b/sql/sql_acl.cc > @@ -7775,6 +7775,8 @@ bool check_grant_column(THD *thd, GRANT_INFO *grant, > table_ref table reference where to check the field > name name of field to check > length length of name > + fld use fld object to check invisibility when it is > + not 0, not_found_field, view_ref_found > > DESCRIPTION > Check the access rights to a column depending on the type of table > @@ -7789,13 +7791,17 @@ bool check_grant_column(THD *thd, GRANT_INFO > *grant, > */ > > bool check_column_grant_in_table_ref(THD *thd, TABLE_LIST * table_ref, > - const char *name, size_t length) > + const char *name, size_t length, > + Field *fld) > { > GRANT_INFO *grant; > const char *db_name; > const char *table_name; > Security_context *sctx= table_ref->security_ctx ? > table_ref->security_ctx : thd->security_ctx; > + if (fld && fld != not_found_field && fld != view_ref_found > + && fld->invisible >= INVISIBLE_SYSTEM) > + return false; > > if (table_ref->view || table_ref->field_translation) > { > @@ -7871,6 +7877,9 @@ bool check_grant_all_columns(THD *thd, ulong > want_access_arg, > > for (; !fields->end_of_fields(); fields->next()) > { > + if (fields->field() && > + fields->field()->invisible >= INVISIBLE_SYSTEM) > + continue; > LEX_CSTRING *field_name= fields->name(); > > if (table_name != fields->get_table_name()) > diff --git a/sql/sql_acl.h b/sql/sql_acl.h > index a608ef0..6da7d4d 100644 > --- a/sql/sql_acl.h > +++ b/sql/sql_acl.h > @@ -239,7 +239,7 @@ bool check_grant_column (THD *thd, GRANT_INFO *grant, > const char *db_name, const char *table_name, > const char *name, size_t length, > Security_context *sctx); > bool check_column_grant_in_table_ref(THD *thd, TABLE_LIST * table_ref, > - const char *name, size_t length); > + const char *name, size_t length, > Field *fld); > bool check_grant_all_columns(THD *thd, ulong want_access, > Field_iterator_table_ref *fields); > bool check_grant_routine(THD *thd, ulong want_access, > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > index 0081365..383341c 100644 > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -5880,7 +5880,7 @@ find_field_in_table_ref(THD *thd, TABLE_LIST > *table_list, > #ifndef NO_EMBEDDED_ACCESS_CHECKS > /* Check if there are sufficient access rights to the found field. */ > if (check_privileges && > - check_column_grant_in_table_ref(thd, *actual_table, name, length)) > + check_column_grant_in_table_ref(thd, *actual_table, name, length, > fld)) > fld= WRONG_GRANT; > else > #endif > @@ -6057,7 +6057,7 @@ find_field_in_tables(THD *thd, Item_ident *item, > #ifndef NO_EMBEDDED_ACCESS_CHECKS > /* Check if there are sufficient access rights to the found field. > */ > if (found && check_privileges && > - check_column_grant_in_table_ref(thd, table_ref, name, length)) > + check_column_grant_in_table_ref(thd, table_ref, name, length, > found)) > found= WRONG_GRANT; > #endif > } > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

2 1

Re: [Maria-developers] Questions regarding closing partitions in MDEV-11084
by Alexey Botchkov 26 Apr '18

26 Apr '18

Hi, Mattias, guys! While investigating the crash, i'd like to discuss that > it seems to close partitions whenever it > is not used in a statement (i.e. require it to be reopened in the next > statement that would use another partition Yes, it does that, handling statements with the specified 'PARTITION' option. The patch supposed to solve the problem when there are too many partitions opened, so i think it must close the unused partitions sometime. No, it doesn't have to happen that often. I planned to check the table_open_cache variable before the forced close. But decided not to do that initially - as it simplified testing, and i thought if someone uses the PARTITION option, he would stick to using this partition anyway. And i forgot about that issue. Best regards. HF 13.04.2018 19:07, Mattias Jonsson wrote: > Hi MariaDB Devs, > > I tried to evaluate spider engine and found an issue where it crashes, > most likely due to MDEV-11084 (Stacktrace and reproducible test case > attached). > > That also leads me to wonder about the performance for partitioned > tables after MDEV-11084, when it seems to close partitions whenever it > is not used in a statement (i.e. require it to be reopened in the next > statement that would use another partition, effectively not using the > open table cache). > > I cannot see anything mentioned in the final commit message hinting on > that it closes partitions not used in the current query, but in the > previous patches it was mentioned without any reason. > > Why does it close the already opened partitions? > https://github.com/MariaDB/server/blob/10.3/sql/ha_partition.cc#L8365 > > I would not mind opening the partitions only when they are to be used > (even though there are engines that need to be tested more), but > closing them makes no sense to me performance wise. Also notice that > the partitions first will be put back into the open table cache and > then on the next query the non-used partitions will be closed and the > needed ones be (re)-opened. > > Regards > Mattias Jonsson

3 2

Re: [Maria-developers] 331034cb6e1: MDEV-11975: SQLCOM_PREPARE of EXPLAIN & ANALYZE statement do not return correct metadata info
by Sergei Golubchik 25 Apr '18

25 Apr '18

Hi, Oleksandr! On Apr 20, Oleksandr Byelkin wrote: > revision-id: 331034cb6e1552e1ab7c12b6f7341f557f76b807 (mariadb-10.3.6-16-g331034cb6e1) > parent(s): cff60be7fe159fdcb2517ce8441610ad512aa7d0 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-04-20 14:40:14 +0200 > message: > > MDEV-11975: SQLCOM_PREPARE of EXPLAIN & ANALYZE statement do not return correct metadata info > > Added metadate info after prepare EXPLAIN/ANALYZE. > > diff --git a/sql/sql_class.cc b/sql/sql_class.cc > index 96485ed15ec..0ec95e11829 100644 > --- a/sql/sql_class.cc > +++ b/sql/sql_class.cc > @@ -2659,18 +2659,32 @@ CHANGED_TABLE_LIST* THD::changed_table_dup(const char *key, size_t key_length) > } > > > -int THD::send_explain_fields(select_result *result, uint8 explain_flags, bool is_analyze) > +void THD::prepare_explain_fields(select_result *result, > + List<Item> *field_list, > + uint8 explain_flags, > + bool is_analyze) > { > - List<Item> field_list; > + ; forgot to remove? otherwise, ok to push... Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] MDEV-13695: INTERSECT precedence is not in line with Oracle even in SQL_MODE=Oracle
by Sergei Golubchik 25 Apr '18

25 Apr '18

Hi, Oleksandr! On Apr 24, Oleksandr Byelkin wrote: > revision-id: cb6aade130b75f3dc4b7c3a1dd4d33864adffd90 (mariadb-10.3.6-43-gcb6aade130b) > parent(s): f79c5a658cc33a10d7744a748a4328254e2cbaf7 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-04-24 12:04:59 +0200 > message: > > MDEV-13695: INTERSECT precedence is not in line with Oracle even in SQL_MODE=Oracle > > Switch off automatic INTERSECT priority for ORACLE MODE ok to push Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] 10.3-MDEV-11953
by Alexander Barkov 25 Apr '18

25 Apr '18

Hello Igor and Sanja, In your branch you stopped to use the global function check_simple_select(). It's not used any more. Please remove it from sql_parse.h and sql_parse.cc. Thanks.

1 0

Re: [Maria-developers] 57673bc: MDEV-14024 PCRE2
by Sergei Golubchik 24 Apr '18

24 Apr '18

Hi, Alexey! On Apr 24, Alexey Botchkov wrote: > > But for mroonga you've just changed the include path. I was > > surprised that it was enough. Perhaps mroonga doesn't use pcre at > > all? > > No, mroonga doesn't use PCRE, just mentions in comfigure scripts. > As far as i can see. At least it compiles fine with my changes. If it doesn't use PCRE, could you remove those lines with the path at all? Would be less confusing. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 57673bc: MDEV-14024 PCRE2
by Sergei Golubchik 23 Apr '18

23 Apr '18

Hi, Alexey! On Apr 23, Alexey Botchkov wrote: > > >> diff --git a/sql/item_cmpfunc.h b/sql/item_cmpfunc.h > >> index 3d11a22..d3f52ed 100644 > >... > >> #define PCRE_STATIC 1 /* Important on Windows */ > >> -#include "pcre.h" /* pcre header file */ > >> +#define PCRE2_CODE_UNIT_WIDTH 8 > > > >you sure, you want it here and not, say, in one universally-included > >headers like my_config.h? You don't define it in other places now, so > >they include different function names and macros. > > Since it's the only place where we include the 'pcre2.h', it probably > makes sence to keep all pcre2-related stuff in one place. No, that was precisely my point - you included pcre2.h in many (more than one) files, but defined PCRE2_CODE_UNIT_WIDTH only here. Otherwise I wouldn't have asked :) > Still i moved the #define to the my_config.h. Right, thanks. > >> +if(EXISTS "${MYSQL_SOURCE_DIR}/pcre2") > >> + set(MYSQL_REGEX_INCLUDE_DIR "${MYSQL_SOURCE_DIR}/pcre2/src") > > > >eh, is that enough? > >there were a lot more changes in the server, than just correcting the > >path. > > Do you mean there's more to be done? Not sure i understand right. To make the server use pcre2 you had to change something in the sources, like, API function names, types names, etc. But for mroonga you've just changed the include path. I was surprised that it was enough. Perhaps mroonga doesn't use pcre at all? Otherwise how could've it compiled without you changing function names and types? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] sql_mode=oracle : Rollback on error
by jerome brauge 23 Apr '18

23 Apr '18

Hello Alexander, I've found one another difference between Mariadb and Oracle : oracle always rollback the active transaction if an error is raised, Maria leave the transaction active. Possible workaround in stored procedure: add an exit handler which do a rollback and resignal the error. What do you think about this ? Regards

4 7

[Maria-developers] Deprecate INT(M) in SHOW?
by Alexander Barkov 23 Apr '18

23 Apr '18

Hello, We have an optional length for integer data types: TINYINT(N) SMALLINT(N) MEDIUMINT(N) INT(N) BIGINT(N) The number given in the parentheses imposes no any limits on the supported range. So for example BIGINT(1) is still a full-featured data type that can store and retrieve any 64-bit number (btw, I find this confusing). If I declare a column without N, it's later displayed in SHOW CREATE TABLE with the default length, so: CREATE TABLE t1 (a INT) becomes CREATE TABLE t1 (a INT(11)); The length is really important only in two cases: 1. In combination with ZEROFILL. 2. When the explicitly specified length is larger than the default length for the data type. For example, INT(40). In this case on cast to string, it's converted to VARCHAR(40) rather than to VARCHAR(11). In all other cases the length does not matter and has no any visible effects. Does anybody see any problems if in 10.4 we fix all metadata statements, such as: - SHOW CREATE TABLE t1 - DESCRIBE t1 - SELECT * FROM INFORMATION_SCHEMA.COLUMNS not to print the default length for non-ZEROFILL columns?

4 5

Re: [Maria-developers] Analysis of the same name of field and name check constraints
by Anel Husakovic 23 Apr '18

23 Apr '18

Hi Vincentiu, Sorry but I have seen now, after sending the mail, that example in this mail is not relevant, please take a look in a example in PR. Regards, Anel On Mon, Apr 23, 2018 at 12:45 PM, Anel Husakovic <anel(a)mariadb.org> wrote: > Hi Vicentiu, > > Regarding the check constraints with *field *and *table *constraint that > have the same name for example: > *`create table t (a int, b check(b>0), constraint b check (b>10));`* > > when new rows are inserted and an error occurs, generated error is not > pointing to exact constraint. For example: > *`insert into t values (-1);`* > ERROR 4025 (23000): CONSTRAINT `b` failed for `test`.`t` > but this means different constraint. > > In order to solve this, here are my opinions: > 1. approach is not to allow this scenario while creating the table, > 2. approach is to generate different message one message for *table *constraint > second for *field *constraint depending where error occurs- and you told > me to go with this one. > > Out of curiosity I also tried with 1. approach problem is that in > *sql/sql_table.cc **mysql_prepare_create_table 4221* > *create/alter_info.check_constraint_list *is tied to table constraints. > However there is another attribute *field_check_constraints *for > *create_info *object but it is always *0*. It is updated in > * sql/unireg.cc* *783*. Problem is that *mysql_prepare_create_table *is > called first and after that *field_check_constraints *is updated,so > there is no mechanism to allow check in early stage for *table*-*field *check > constraint. > After that there is a validation according to the each constraint in > *sql/table.cc* *verify_constraints 5232* - and this is 2. approach. > > I finished 2. approach and what I have that each test already written in > main directory failed because of different error message for a specific > case. > So here is a patch for 2. approach: > https://github.com/MariaDB/server/pull/715 > > In this approach we are handling each error independently. > Also we need to change each test regarding the ER_CONSTRAINT_FAILED where > should be raised newly created error ER_FIELD_CONSTRAINT_FAILED. > > > Also I have updated PR *IS.check_constraint *, rebased everything in one > file > https://github.com/MariaDB/server/pull/689/ > > Best regards, > Anel > > > >

1 0

Re: [Maria-developers] [Commits] a20e1ce85ec: MDEV-13232: Assertion `(&(&share->intern_lock)->m_mutex)->count > 0 && pthread_equal(pthread_self(), (&(&share->intern_lock)->m_mutex)->thread)' failed in _ma_state_info_write
by Alexander Barkov 23 Apr '18

23 Apr '18

Hi Sanja, the patch is OK to push. Thanks. On 04/23/2018 12:37 PM, Oleksandr Byelkin wrote: > revision-id: a20e1ce85ec56616fa048e9ba39c25c65b098fce (mariadb-10.3.6-36-ga20e1ce85ec) > parent(s): c6ba758d1d41c11466b8f9b61b4546efc95aa689 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-04-23 10:35:33 +0200 > message: > > MDEV-13232: Assertion `(&(&share->intern_lock)->m_mutex)->count > 0 && pthread_equal(pthread_self(), (&(&share->intern_lock)->m_mutex)->thread)' failed in _ma_state_info_write > > Limit length of result of "negative" operation to something reasonable > > --- > mysql-test/main/union.result | 14 ++++++++++++++ > mysql-test/main/union.test | 15 +++++++++++++++ > sql/item_func.cc | 3 +++ > 3 files changed, 32 insertions(+) > > diff --git a/mysql-test/main/union.result b/mysql-test/main/union.result > index 4e5f9312e03..39cd68ae851 100644 > --- a/mysql-test/main/union.result > +++ b/mysql-test/main/union.result > @@ -2500,5 +2500,19 @@ t2 CREATE TABLE `t2` ( > DROP TABLE t2; > DROP TABLE t1; > # > +# MDEV-13232: Assertion `(&(&share->intern_lock)->m_mutex)->count > 0 && > +# pthread_equal(pthread_self(), (&(&share->intern_lock)->m_mutex)-> > +# thread)' failed in _ma_state_info_write > +# > +CREATE TABLE t1 (c1 CHAR(8)); > +INSERT INTO t1 VALUES ('10'),('-10'); > +CREATE TABLE t2 (c2 CHAR); > +SET @a= CAST('10' AS CHAR); > +SELECT c1 FROM t1 UNION SELECT - @a FROM t2; > +c1 > +10 > +-10 > +drop table t1,t2; > +# > # End of 10.3 tests > # > diff --git a/mysql-test/main/union.test b/mysql-test/main/union.test > index f86cae87524..b9b38271f83 100644 > --- a/mysql-test/main/union.test > +++ b/mysql-test/main/union.test > @@ -1745,6 +1745,21 @@ SHOW CREATE TABLE t2; > DROP TABLE t2; > DROP TABLE t1; > > +--echo # > +--echo # MDEV-13232: Assertion `(&(&share->intern_lock)->m_mutex)->count > 0 && > +--echo # pthread_equal(pthread_self(), (&(&share->intern_lock)->m_mutex)-> > +--echo # thread)' failed in _ma_state_info_write > +--echo # > + > +CREATE TABLE t1 (c1 CHAR(8)); > +INSERT INTO t1 VALUES ('10'),('-10'); > + > +CREATE TABLE t2 (c2 CHAR); > +SET @a= CAST('10' AS CHAR); > + > +SELECT c1 FROM t1 UNION SELECT - @a FROM t2; > + > +drop table t1,t2; > > --echo # > --echo # End of 10.3 tests > diff --git a/sql/item_func.cc b/sql/item_func.cc > index e2740272385..ae6c61e83f4 100644 > --- a/sql/item_func.cc > +++ b/sql/item_func.cc > @@ -1935,6 +1935,9 @@ void Item_func_neg::fix_length_and_dec_double() > set_handler(&type_handler_double); > decimals= args[0]->decimals; // Preserve NOT_FIXED_DEC > max_length= args[0]->max_length + 1; > + // Limit length with something reasonable > + uint32 mlen= type_handler()->max_display_length(this); > + set_if_smaller(max_length, mlen); > unsigned_flag= false; > } > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits >

1 0

Re: [Maria-developers] ce222a7: MDEV-15915 Add Feature_json status variable.
by Sergei Golubchik 23 Apr '18

23 Apr '18

Hi, Alexey! On Apr 23, Alexey Botchkov wrote: > > Elena commented in MDEV about "Feature_json" being possibly confusing. > I've seen Elena's comment. Can also add GEOJSON functions to her list. > Still i can't figure out anything better than Feature_json as the name > for this variable. > The self-explaining name is either too long or possible misleading. Feature_json_functions ? But I think Feature_json is fine too, one hardly thinks of "EXPLAIN JSON" when talking about "JSON support". Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] ce222a7: MDEV-15915 Add Feature_json status variable.
by Sergei Golubchik 22 Apr '18

22 Apr '18

Hi, Alexey! On Apr 22, Alexey Botchkov wrote: > revision-id: ce222a71bf75b2125cc588371a3debb3e4a22d7f (mariadb-10.3.6-34-gce222a7) > parent(s): 9f84451d87ba1924539b7d2ab69619b71d0155b9 > committer: Alexey Botchkov > timestamp: 2018-04-22 16:01:18 +0400 > message: > > MDEV-15915 Add Feature_json status variable. > > --- > mysql-test/main/status.test | 7 +++++++ > sql/item_create.cc | 26 ++++++++++++++++++++++++++ > sql/mysqld.cc | 1 + > sql/sql_class.h | 1 + > 4 files changed, 35 insertions(+) > > diff --git a/sql/item_create.cc b/sql/item_create.cc > index 901dfa0..8b8476b 100644 > --- a/sql/item_create.cc > +++ b/sql/item_create.cc > @@ -5118,6 +5118,7 @@ Create_func_json_exists Create_func_json_exists::s_singleton; > Item* > Create_func_json_exists::create_2_arg(THD *thd, Item *arg1, Item *arg2) > { > + status_var_increment(current_thd->status_var.feature_json); > return new (thd->mem_root) Item_func_json_exists(thd, arg1, arg2); I believe you've added that to all JSON functions, I didn't try to double-check that :) > } > > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index 6e1a599..ae14c80 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -8553,6 +8553,7 @@ SHOW_VAR status_vars[]= { > {"Feature_fulltext", (char*) offsetof(STATUS_VAR, feature_fulltext), SHOW_LONG_STATUS}, > {"Feature_gis", (char*) offsetof(STATUS_VAR, feature_gis), SHOW_LONG_STATUS}, > {"Feature_invisible_columns", (char*) offsetof(STATUS_VAR, feature_invisible_columns), SHOW_LONG_STATUS}, > + {"Feature_json", (char*) offsetof(STATUS_VAR, feature_json), SHOW_LONG_STATUS}, > {"Feature_locale", (char*) offsetof(STATUS_VAR, feature_locale), SHOW_LONG_STATUS}, > {"Feature_subquery", (char*) offsetof(STATUS_VAR, feature_subquery), SHOW_LONG_STATUS}, > {"Feature_timezone", (char*) offsetof(STATUS_VAR, feature_timezone), SHOW_LONG_STATUS}, Elena commented in MDEV about "Feature_json" being possibly confusing. Does your using "Feature_json" name mean that * you considered it and intentionally kept the old name, thinking it's fine or * you didn't see the comment, and now you might reconsider? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 67260fd: MDEV-15130 Assertion `table->s->null_bytes == 0' failed in ...
by Sergei Golubchik 20 Apr '18

20 Apr '18

Hi, Sachin! On Apr 19, sachin wrote: > diff --git a/storage/perfschema/ha_perfschema.cc b/storage/perfschema/ha_perfschema.cc > index e8e9581..d5846f2 100644 > --- a/storage/perfschema/ha_perfschema.cc > +++ b/storage/perfschema/ha_perfschema.cc > @@ -64,6 +64,16 @@ static int pfs_discover_table(handlerton *hton, THD *thd, TABLE_SHARE *share) > { > const PFS_engine_table_share *pfs_share; > > + DBUG_EXECUTE_IF("test_pseudo_invisible", { > + my_error(ER_INTERNAL_ERROR, MYF(0), "Don't query performance_schema when" > + "test_pseudo_invisible is on"); > + return 1; > + }); > + DBUG_EXECUTE_IF("test_completely_invisible", { > + my_error(ER_INTERNAL_ERROR, MYF(0), "Don't query performance_schema when" > + "test_completely_invisible is on"); > + return 1; > + }); > if ((pfs_share= find_table_share(share->db.str, share->table_name.str))) > return share->init_from_sql_statement_string(thd, false, > pfs_share->sql.str, Better put it inside TABLE_SHARE::init_from_sql_statement_string to cover Connect and FederatedX too. Or - I'd say it's an option too - just close it as not a bug. DBUG_EXECUTE_IF can cause a crash, can make a server to misbehave, etc. That's normal, it's a debug hack that intentionally makes server to do something weird. I wouldn't spend too much time on making it user friendly and error-proof. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 1

Re: [Maria-developers] 346c8ab9533: MDEV-11975: SQLCOM_PREPARE of EXPLAIN & ANALYZE statement do not return correct metadata info
by Sergei Golubchik 20 Apr '18

20 Apr '18

Hi, Oleksandr! On Apr 18, Oleksandr Byelkin wrote: > revision-id: 346c8ab9533a3de6a4cb348428402ffee2aa8da2 (mariadb-10.3.6-16-g346c8ab9533) > parent(s): cff60be7fe159fdcb2517ce8441610ad512aa7d0 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-04-18 19:34:12 +0200 > message: > > MDEV-11975: SQLCOM_PREPARE of EXPLAIN & ANALYZE statement do not return correct metadata info > > Added metadate info after prepare EXPLAIN/ANALYZE. > > --- > mysql-test/main/mysql_client_test.result | 122 ++++++++++++ > mysql-test/main/mysql_client_test.test | 7 +- > sql/sql_class.cc | 18 +- > sql/sql_class.h | 3 +- > sql/sql_explain.cc | 3 +- > sql/sql_parse.cc | 5 +- > sql/sql_prepare.cc | 24 ++- > sql/sql_prepare.h | 2 + > tests/mysql_client_test.c | 323 ++++++++++++++++++++++++++++++- > 9 files changed, 493 insertions(+), 14 deletions(-) > > diff --git a/mysql-test/main/mysql_client_test.result b/mysql-test/main/mysql_client_test.result > index 83ef8d442b3..20385acfa00 100644 > --- a/mysql-test/main/mysql_client_test.result > +++ b/mysql-test/main/mysql_client_test.result > @@ -122,5 +122,127 @@ EOF > mysql_stmt_next_result(): 0; field_count: 0 > # ------------------------------------ > > +# cat MYSQL_TMP_DIR/test_explain_meta.out.log > +# ------------------------------------ okay, altought I'd rather add prepare metadata output to mysqltest. Like, either print prepare metadata on --enable_metadata or introduce a new command --enable_prepare_metadata > +SELECT number of fields: 1 > +EXPALIN number of fields: 10 > + - 0: name: 'id'/''; table: ''/''; db: ''; catalog: 'def'; length: 3; max_length: 0; type: 8; decimals: 0 > + - 1: name: 'select_type'/''; table: ''/''; db: ''; catalog: 'def'; length: 57; max_length: 0; type: 253; decimals: 39 > + - 2: name: 'table'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 3: name: 'type'/''; table: ''/''; db: ''; catalog: 'def'; length: 30; max_length: 0; type: 253; decimals: 39 > + - 4: name: 'possible_keys'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 5: name: 'key'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 6: name: 'key_len'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 7: name: 'ref'/''; table: ''/''; db: ''; catalog: 'def'; length: 6144; max_length: 0; type: 253; decimals: 39 > + - 8: name: 'rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 10; max_length: 0; type: 8; decimals: 0 > + - 9: name: 'Extra'/''; table: ''/''; db: ''; catalog: 'def'; length: 765; max_length: 0; type: 253; decimals: 39 > +EXPALIN JSON number of fields: 1 > + - 0: name: 'EXPLAIN'/''; table: ''/''; db: ''; catalog: 'def'; length: 234; max_length: 0; type: 253; decimals: 39 > +ANALYZE number of fields: 13 > + - 0: name: 'id'/''; table: ''/''; db: ''; catalog: 'def'; length: 3; max_length: 0; type: 8; decimals: 0 > + - 1: name: 'select_type'/''; table: ''/''; db: ''; catalog: 'def'; length: 57; max_length: 0; type: 253; decimals: 39 > + - 2: name: 'table'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 3: name: 'type'/''; table: ''/''; db: ''; catalog: 'def'; length: 30; max_length: 0; type: 253; decimals: 39 > + - 4: name: 'possible_keys'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 5: name: 'key'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 6: name: 'key_len'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 7: name: 'ref'/''; table: ''/''; db: ''; catalog: 'def'; length: 6144; max_length: 0; type: 253; decimals: 39 > + - 8: name: 'rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 10; max_length: 0; type: 8; decimals: 0 > + - 9: name: 'r_rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 10 > + - 10: name: 'filtered'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 2 > + - 11: name: 'r_filtered'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 2 > + - 12: name: 'Extra'/''; table: ''/''; db: ''; catalog: 'def'; length: 765; max_length: 0; type: 253; decimals: 39 > +ANALYZE JSON number of fields: 1 > + - 0: name: 'ANALYZE'/''; table: ''/''; db: ''; catalog: 'def'; length: 234; max_length: 0; type: 253; decimals: 39 > +EXPALIN INSERT number of fields: 10 > + - 0: name: 'id'/''; table: ''/''; db: ''; catalog: 'def'; length: 3; max_length: 0; type: 8; decimals: 0 > + - 1: name: 'select_type'/''; table: ''/''; db: ''; catalog: 'def'; length: 57; max_length: 0; type: 253; decimals: 39 > + - 2: name: 'table'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 3: name: 'type'/''; table: ''/''; db: ''; catalog: 'def'; length: 30; max_length: 0; type: 253; decimals: 39 > + - 4: name: 'possible_keys'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 5: name: 'key'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 6: name: 'key_len'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 7: name: 'ref'/''; table: ''/''; db: ''; catalog: 'def'; length: 6144; max_length: 0; type: 253; decimals: 39 > + - 8: name: 'rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 10; max_length: 0; type: 8; decimals: 0 > + - 9: name: 'Extra'/''; table: ''/''; db: ''; catalog: 'def'; length: 765; max_length: 0; type: 253; decimals: 39 > +EXPALIN JSON INSERT number of fields: 1 > + - 0: name: 'EXPLAIN'/''; table: ''/''; db: ''; catalog: 'def'; length: 234; max_length: 0; type: 253; decimals: 39 > +ANALYZE INSERT number of fields: 13 > + - 0: name: 'id'/''; table: ''/''; db: ''; catalog: 'def'; length: 3; max_length: 0; type: 8; decimals: 0 > + - 1: name: 'select_type'/''; table: ''/''; db: ''; catalog: 'def'; length: 57; max_length: 0; type: 253; decimals: 39 > + - 2: name: 'table'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 3: name: 'type'/''; table: ''/''; db: ''; catalog: 'def'; length: 30; max_length: 0; type: 253; decimals: 39 > + - 4: name: 'possible_keys'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 5: name: 'key'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 6: name: 'key_len'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 7: name: 'ref'/''; table: ''/''; db: ''; catalog: 'def'; length: 6144; max_length: 0; type: 253; decimals: 39 > + - 8: name: 'rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 10; max_length: 0; type: 8; decimals: 0 > + - 9: name: 'r_rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 10 > + - 10: name: 'filtered'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 2 > + - 11: name: 'r_filtered'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 2 > + - 12: name: 'Extra'/''; table: ''/''; db: ''; catalog: 'def'; length: 765; max_length: 0; type: 253; decimals: 39 > +ANALYZE JSON INSERT number of fields: 1 > + - 0: name: 'ANALYZE'/''; table: ''/''; db: ''; catalog: 'def'; length: 234; max_length: 0; type: 253; decimals: 39 > +EXPALIN UPDATE number of fields: 10 > + - 0: name: 'id'/''; table: ''/''; db: ''; catalog: 'def'; length: 3; max_length: 0; type: 8; decimals: 0 > + - 1: name: 'select_type'/''; table: ''/''; db: ''; catalog: 'def'; length: 57; max_length: 0; type: 253; decimals: 39 > + - 2: name: 'table'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 3: name: 'type'/''; table: ''/''; db: ''; catalog: 'def'; length: 30; max_length: 0; type: 253; decimals: 39 > + - 4: name: 'possible_keys'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 5: name: 'key'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 6: name: 'key_len'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 7: name: 'ref'/''; table: ''/''; db: ''; catalog: 'def'; length: 6144; max_length: 0; type: 253; decimals: 39 > + - 8: name: 'rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 10; max_length: 0; type: 8; decimals: 0 > + - 9: name: 'Extra'/''; table: ''/''; db: ''; catalog: 'def'; length: 765; max_length: 0; type: 253; decimals: 39 > +EXPALIN JSON UPDATE number of fields: 1 > + - 0: name: 'EXPLAIN'/''; table: ''/''; db: ''; catalog: 'def'; length: 234; max_length: 0; type: 253; decimals: 39 > +ANALYZE UPDATE number of fields: 13 > + - 0: name: 'id'/''; table: ''/''; db: ''; catalog: 'def'; length: 3; max_length: 0; type: 8; decimals: 0 > + - 1: name: 'select_type'/''; table: ''/''; db: ''; catalog: 'def'; length: 57; max_length: 0; type: 253; decimals: 39 > + - 2: name: 'table'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 3: name: 'type'/''; table: ''/''; db: ''; catalog: 'def'; length: 30; max_length: 0; type: 253; decimals: 39 > + - 4: name: 'possible_keys'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 5: name: 'key'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 6: name: 'key_len'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 7: name: 'ref'/''; table: ''/''; db: ''; catalog: 'def'; length: 6144; max_length: 0; type: 253; decimals: 39 > + - 8: name: 'rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 10; max_length: 0; type: 8; decimals: 0 > + - 9: name: 'r_rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 10 > + - 10: name: 'filtered'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 2 > + - 11: name: 'r_filtered'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 2 > + - 12: name: 'Extra'/''; table: ''/''; db: ''; catalog: 'def'; length: 765; max_length: 0; type: 253; decimals: 39 > +ANALYZE JSON UPDATE number of fields: 1 > + - 0: name: 'ANALYZE'/''; table: ''/''; db: ''; catalog: 'def'; length: 234; max_length: 0; type: 253; decimals: 39 > +EXPALIN DELETE number of fields: 10 > + - 0: name: 'id'/''; table: ''/''; db: ''; catalog: 'def'; length: 3; max_length: 0; type: 8; decimals: 0 > + - 1: name: 'select_type'/''; table: ''/''; db: ''; catalog: 'def'; length: 57; max_length: 0; type: 253; decimals: 39 > + - 2: name: 'table'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 3: name: 'type'/''; table: ''/''; db: ''; catalog: 'def'; length: 30; max_length: 0; type: 253; decimals: 39 > + - 4: name: 'possible_keys'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 5: name: 'key'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 6: name: 'key_len'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 7: name: 'ref'/''; table: ''/''; db: ''; catalog: 'def'; length: 6144; max_length: 0; type: 253; decimals: 39 > + - 8: name: 'rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 10; max_length: 0; type: 8; decimals: 0 > + - 9: name: 'Extra'/''; table: ''/''; db: ''; catalog: 'def'; length: 765; max_length: 0; type: 253; decimals: 39 > +EXPALIN JSON DELETE number of fields: 1 > + - 0: name: 'EXPLAIN'/''; table: ''/''; db: ''; catalog: 'def'; length: 234; max_length: 0; type: 253; decimals: 39 > +ANALYZE DELETE number of fields: 13 > + - 0: name: 'id'/''; table: ''/''; db: ''; catalog: 'def'; length: 3; max_length: 0; type: 8; decimals: 0 > + - 1: name: 'select_type'/''; table: ''/''; db: ''; catalog: 'def'; length: 57; max_length: 0; type: 253; decimals: 39 > + - 2: name: 'table'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 3: name: 'type'/''; table: ''/''; db: ''; catalog: 'def'; length: 30; max_length: 0; type: 253; decimals: 39 > + - 4: name: 'possible_keys'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 5: name: 'key'/''; table: ''/''; db: ''; catalog: 'def'; length: 192; max_length: 0; type: 253; decimals: 39 > + - 6: name: 'key_len'/''; table: ''/''; db: ''; catalog: 'def'; length: 12288; max_length: 0; type: 253; decimals: 39 > + - 7: name: 'ref'/''; table: ''/''; db: ''; catalog: 'def'; length: 6144; max_length: 0; type: 253; decimals: 39 > + - 8: name: 'rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 10; max_length: 0; type: 8; decimals: 0 > + - 9: name: 'r_rows'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 10 > + - 10: name: 'filtered'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 2 > + - 11: name: 'r_filtered'/''; table: ''/''; db: ''; catalog: 'def'; length: 4; max_length: 0; type: 5; decimals: 2 > + - 12: name: 'Extra'/''; table: ''/''; db: ''; catalog: 'def'; length: 765; max_length: 0; type: 253; decimals: 39 > +ANALYZE JSON DELETE number of fields: 1 > + - 0: name: 'ANALYZE'/''; table: ''/''; db: ''; catalog: 'def'; length: 234; max_length: 0; type: 253; decimals: 39 > +# ------------------------------------ > + > + > SET @@global.general_log= @old_general_log; > SET @@global.slow_query_log= @old_slow_query_log; > diff --git a/sql/sql_class.cc b/sql/sql_class.cc > index 96485ed15ec..349ec576264 100644 > --- a/sql/sql_class.cc > +++ b/sql/sql_class.cc > @@ -2659,18 +2660,27 @@ CHANGED_TABLE_LIST* THD::changed_table_dup(const char *key, size_t key_length) > } > > > -int THD::send_explain_fields(select_result *result, uint8 explain_flags, bool is_analyze) > +int THD::send_explain_fields(select_result *result, > + uint8 explain_flags, > + bool is_analyze, > + Prepared_statement *stmt) > { > List<Item> field_list; > + int rc; > if (lex->explain_json) > make_explain_json_field_list(field_list, is_analyze); > else > make_explain_field_list(field_list, explain_flags, is_analyze); > > result->prepare(field_list, NULL); > - return (result->send_result_set_metadata(field_list, > - Protocol::SEND_NUM_ROWS | > - Protocol::SEND_EOF)); > + if (stmt) > + rc= send_prep_stmt(stmt, result->field_count(field_list)) || > + result->send_result_set_metadata(field_list, Protocol::SEND_EOF); > + else > + rc= result->send_result_set_metadata(field_list, > + Protocol::SEND_NUM_ROWS | > + Protocol::SEND_EOF); > + return(rc); > } dunno, I'd rather keep prepare functionality in sql_prepare.cc. Like, void THD::prepare_explain_fields(select_result *result, List<Item> *field_list, uint8 explain_flags, bool is_analyze) { if (lex->explain_json) make_explain_json_field_list(field_list, is_analyze); else make_explain_field_list(field_list, explain_flags, is_analyze); result->prepare(field_list, NULL); } int THD::send_explain_fields(select_result *result, uint8 explain_flags, bool is_analyze) { List<Item> field_list; prepare_explain_fields(result, field_list, explain_flags, is_analyze); return result->send_result_set_metadata(field_list, Protocol::SEND_NUM_ROWS | Protocol::SEND_EOF); } and in sql_prepare.cc you do List<Item> field_list; thd->prepare_explain_fields(result, field_list, explain_flags, is_analyze); return send_prep_stmt(stmt, result->field_count(field_list)) || result->send_result_set_metadata(field_list, Protocol::SEND_EOF); Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 3

Re: [Maria-developers] [Commits] 602865f9f24: MDEV-13727 rpl.rpl_concurrency_error failed
by Alexander Barkov 20 Apr '18

20 Apr '18

Hello Andrei, In LOGGER::slow_log_print() there is a piece of this code: query_utime= (current_utime - thd->start_utime); lock_utime= (thd->utime_after_lock - thd->start_utime); my_hrtime_t current_time= { hrtime_from_time(thd->start_time) + thd->start_time_sec_part + query_utime }; If we don't reset thd->start_utime in the beginning of sp_instr_stmt::execute(), then this code will calculate something useless. It seems that this function: bool LOGGER::slow_log_print( THD *thd, const char *query, size_t query_length, ulonglong current_utime) should not use thd->start_time, thd->start_utime, thd->start_time_sec_part. and should get this information from the argument: bool LOGGER::slow_log_print( THD *thd, const char *query, size_t query_length, const QUERY_START_TIME_INFO *start, ulonglong current_utime) And sp_instr_stmt::execute() should not do thd->set_time(). The whole logic should reside in sql_parse.cc. 1. Remember the current time 2. Execute the statement 3. Log the query if it was slow, using the time remembered at step 1. Is my understanding correct? Thanks. On 04/19/2018 03:42 PM, andrei.elkin(a)pp.inet.fi wrote: > revision-id: 602865f9f242a7ed5e3679af60a78698c03d93f5 (mariadb-10.3.6-14-g602865f9f24) > parent(s): 02e897ca57168059c6a74d1129158afea93aa875 > author: Andrei Elkin > committer: Andrei Elkin > timestamp: 2018-04-18 13:04:46 +0300 > message: > > MDEV-13727 rpl.rpl_concurrency_error failed > > The test actually revealed a flaw in MDEV-8305 > which inadvertently enrolled the trigger and > stored function into slow query reporting aimed exclusively > to the stored procedure. > Specifically to the test, a query on the master was logged > with a timestamp of the query's top-level statement but its (post > update) trigger computed one more (later) timestamp which got > inserted into another table. > > Master-vs-slave discrepancy became evident thanks to > different execution time of the trigger combined with the fact of the > logged master timestamp in the micro-second fraction format was > truncated on the slave. The latter means the slave execution of the query > always started earlier than on the master as far as the timestamp was concerned. > And when the fractional part was close to 1 the 2nd timestamp acquired by > MDEV-8305 piece of logics could overflow to next integer value. > That's how the master table appeared to be temporarly behind the slave's one. > > Fixed with restoring the pre-MDEV-8305 behavior not to reset > the current statement's timestamp when the query is not from a stored procedure. > That is the trigger's or stored function's queries continue to use > the top-level statement's timestamp as they are supposed to. > > --- > mysql-test/main/func_time.result | 26 ++++++++++++++++++ > mysql-test/main/func_time.test | 59 ++++++++++++++++++++++++++++++++++++++++ > sql/sp_head.cc | 4 +-- > 3 files changed, 87 insertions(+), 2 deletions(-) > > diff --git a/mysql-test/main/func_time.result b/mysql-test/main/func_time.result > index 2772f850ce9..9babfc6a48b 100644 > --- a/mysql-test/main/func_time.result > +++ b/mysql-test/main/func_time.result > @@ -3484,3 +3484,29 @@ t1 CREATE TABLE `t1` ( > `c5` varchar(100) DEFAULT NULL > ) ENGINE=MyISAM DEFAULT CHARSET=latin1 > DROP TABLE t1; > +CREATE TABLE t_ts (a timestamp(6)); > +CREATE TABLE t_trig (a timestamp(6)); > +CREATE FUNCTION fn_sleep_before_now() returns int > +BEGIN > +INSERT INTO t_ts SET a= current_timestamp(6); > +RETURN 0; > +END// > +CREATE TRIGGER trg_insert_t_ts after INSERT on t_ts for each row > +BEGIN > +INSERT into t_trig set a= current_timestamp(6); > +END// > +SET @sav_slow_query_log= @@session.slow_query_log; > +SET @@session.slow_query_log= ON; > +SELECT current_timestamp(6),fn_sleep_before_now() INTO @ts_cur, @ts_func; > +SELECT a FROM t_ts LIMIT 1 into @ts_func; > +SELECT a FROM t_trig LIMIT 1 into @ts_trig; > +DELETE FROM t_ts; > +DELETE FROM t_trig; > +SET @@session.slow_query_log= OFF; > +SELECT current_timestamp(6),fn_sleep_before_now() INTO @ts_cur, @func_ts; > +SELECT a FROM t_ts LIMIT 1 into @ts_func; > +SELECT a FROM t_trig LIMIT 1 into @ts_trig; > +SET @@session.slow_query_log= @sav_slow_query_log; > +DROP FUNCTION fn_sleep_before_now; > +DROP TRIGGER trg_insert_t_ts; > +DROP TABLE t_ts, t_trig; > diff --git a/mysql-test/main/func_time.test b/mysql-test/main/func_time.test > index 5417cb20a92..ad207218969 100644 > --- a/mysql-test/main/func_time.test > +++ b/mysql-test/main/func_time.test > @@ -2057,3 +2057,62 @@ EXECUTE IMMEDIATE > USING NULL, '10', 10, 10.0, 10e0, TIME'10:20:30'; > SHOW CREATE TABLE t1; > DROP TABLE t1; > + > + > +############ > +# MDEV-13727 > +# Current timestamp functions inside stored functions must return the > +# value of the top-level statement's timestamp (its start time). > +# This must hold regardless of @@slow_query_log option. > +# > + > +CREATE TABLE t_ts (a timestamp(6)); > +CREATE TABLE t_trig (a timestamp(6)); > +delimiter //; > +CREATE FUNCTION fn_sleep_before_now() returns int > +BEGIN > + INSERT INTO t_ts SET a= current_timestamp(6); > + RETURN 0; > +END// > +CREATE TRIGGER trg_insert_t_ts after INSERT on t_ts for each row > +BEGIN > + INSERT into t_trig set a= current_timestamp(6); > +END// > +delimiter ;// > + > +SET @sav_slow_query_log= @@session.slow_query_log; > + > +# @@slow_query_log ON check > +SET @@session.slow_query_log= ON; > +SELECT current_timestamp(6),fn_sleep_before_now() INTO @ts_cur, @ts_func; > + > +SELECT a FROM t_ts LIMIT 1 into @ts_func; > +SELECT a FROM t_trig LIMIT 1 into @ts_trig; > +if (!`SELECT @ts_cur = @ts_func and @ts_func = @ts_trig`) > +{ > + SELECT @ts_cur, @ts_func, @ts_trig; > + --die Error: timestamps must be equal but they diverge > +} > +DELETE FROM t_ts; > +DELETE FROM t_trig; > + > +# @@slow_query_log OFF check > +SET @@session.slow_query_log= OFF; > +SELECT current_timestamp(6),fn_sleep_before_now() INTO @ts_cur, @func_ts; > +SELECT a FROM t_ts LIMIT 1 into @ts_func; > +SELECT a FROM t_trig LIMIT 1 into @ts_trig; > +if (!`SELECT @ts_cur = @ts_func and @ts_func = @ts_trig`) > +{ > + SELECT @ts_cur, @ts_func, @ts_trig; > + --die Error: timestamps must be equal but they diverge > +} > + > +# Cleanup > +SET @@session.slow_query_log= @sav_slow_query_log; > +DROP FUNCTION fn_sleep_before_now; > +DROP TRIGGER trg_insert_t_ts; > +DROP TABLE t_ts, t_trig; > + > +# > +# End of MDEV-13727 > +################### > diff --git a/sql/sp_head.cc b/sql/sp_head.cc > index f7847bae89d..90fe77e6158 100644 > --- a/sql/sp_head.cc > +++ b/sql/sp_head.cc > @@ -3471,7 +3471,7 @@ sp_instr_stmt::execute(THD *thd, uint *nextp) > thd->profiling.set_query_source(m_query.str, m_query.length); > #endif > > - if ((save_enable_slow_log= thd->enable_slow_log)) > + if ((save_enable_slow_log= thd->enable_slow_log) && !thd->in_sub_stmt) > { > /* > Save start time info for the CALL statement and overwrite it with the > @@ -3544,7 +3544,7 @@ sp_instr_stmt::execute(THD *thd, uint *nextp) > } > } > /* Restore the original query start time */ > - if (thd->enable_slow_log) > + if (thd->enable_slow_log && !thd->in_sub_stmt) > thd->restore_query_start_time(&time_info); > > DBUG_RETURN(res || thd->is_error()); > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits >

1 0

Re: [Maria-developers] Mdev-10664 Add statuses about optimistic parallel replication stalls.
by Sachin Setiya 19 Apr '18

19 Apr '18

Hi Kristian! Sorry for late reply , I was on vacations. On Tue, Jan 10, 2017 at 7:31 PM, Kristian Nielsen <knielsen(a)knielsen-hq.org> wrote: > Sachin Setiya <sachin.setiya(a)mariadb.com> writes: > > >> Right, but you still need _some_ way to synchronise multiple threads > > > I used atomic builtins my_atomic_add64_explicit. > > That seems fine (but see comment below). > > > Documentation:- > > > > Slave_DDL_Events:- > > > > Description: Number of DDL statements slave have executed in > > replication channel X. > > Scope: Per Replication Channel > > Data Type: numeric > > Introduced: MariaDB 10.1 > > > > Slave_Non_Transactional_Events:- > > > > Description: Number of Non Transactional "no idea what should be the > > next word " slave have executed in replication channel X. > > Scope: Per Replication Channel > > Data Type: numeric > > Introduced: MariaDB 10.1 > > I checked, the term "event group" is used already multiple places in the > documentation, so I think it can be used here also. An event group can be a > transaction (possibly with multiple transactional DML statements), or a DDL > or non-transactional statement. > > What you are doing is introducing status variables to help understand > optimistic (and aggressive) parallel replication modes. > > In optimistic parallel replication, a non-transactional event group is not > safe to run in parallel with other non-transactional events. So that event > is made to wait for all prior event groups to reach commit state, before > being allowed to execute. However, other transactional events are not > delayed by the non-transactional event group. > > Slave_Non_Transactional_Events counts the occurences of this. > > Then in case of a DDL statement, this can potentially be unsafe to run in > parallel even with other transactional events. So for a DDL, the DDL has to > wait for all prior event groups, _and_ subsequent event groups have to wait > for the DDL. > > Slave_DDL_Events counts these occurences. > > This is what the documentation, and commit messages, should explain, I > think. > > I think the status variables describe the values for > @@default_master_connection?, this should also be explained in the > documentation. > > Why did you decide to put this information into a status variable? > Normally, > slave status is seen in SHOW SLAVE STATUS, which supports showing status > for > a particular connection without using @@default_master_connection. > > Sorry for this , I guess It was a confusion I was looking at Slave_Running this is avaliable in show status as well as show slave status. So I thought for if I want to show some information in SHOW slave status this has to be in show status. And this is wrong. Now we can see non trans events / ddl events in show slave status. Show I remove this vars from Show Status, or it is okay ? > > diff --git a/sql/log_event.h b/sql/log_event.h > > index 90900f6..579607f 100644 > > --- a/sql/log_event.h > > +++ b/sql/log_event.h > > @@ -50,6 +50,7 @@ > > #endif > > > > #include "rpl_gtid.h" > > +#include "my_atomic.h" > > Why include my_atomic.h in the header file, when you only use it in the .cc > file? > Changed. > > > + mysql_mutex_lock(&LOCK_active_mi); > > + if (master_info_index) > > + { > > + mi= master_info_index-> > > + get_master_info(&thd->variables.default_master_connection, > > + Sql_condition::WARN_LEVEL_NOTE); > > + if (mi) > > + tmp= mi->total_ddl_events; > > > + if (mi) > > + tmp= mi->total_non_trans_events; > > Better use an atomic primitive here to read the value. Taking a mutex does > not help synchronise against my_atomic_add64_explicit(). > > Changed. > > + } > > + mysql_mutex_unlock(&LOCK_active_mi); > > + if (mi) > > + *((longlong *)buff)= tmp; > > + else > > + var->type= SHOW_UNDEF; > > + > > + return 0; > > +} > > #endif /* HAVE_REPLICATION */ > > > > static int show_open_tables(THD *thd, SHOW_VAR *var, char *buff, > > > index 9365c06..a474aa5 100644 > > --- a/sql/rpl_mi.h > > +++ b/sql/rpl_mi.h > > @@ -303,6 +303,12 @@ class Master_info : public > Slave_reporting_capability > > > > /* The parallel replication mode. */ > > enum_slave_parallel_mode parallel_mode; > > + > > + /* No of DDL statements */ > > + long total_ddl_events; > > + > > + /* No of non-transactional statements*/ > > + long total_non_trans_events; > > "long" is not the appropriate type here. Certainly not since you are using > my_atomic_add64_explicit(). Use a type that is the same size on all > platforms (and why use a signed type?). > > Changed to uint64. > - Kristian. > Attached a newer version of patch , please review it. -- Regards Sachin

3 7

[Maria-developers] CREATE SEQUENCE is coming
by Michael Widenius 18 Apr '18

18 Apr '18

Hi! For those that are trying to port applications to MariaDB from other databases or need more features than the current AUTO_INCREMENT provides, the following may be of interest: First version of sequence is pushed to bb-10.2-sequence. It should be merged to bb-10.2-compatilbity and then to 10.3 shortly (1-2 weeks). What is working for the moment: CREATE OR REPLACE [TEMPORARY] SEQUENCE [IF NOT EXISTS] name [ INCREMENT [ BY | = ] increment ] [ MINVALUE [=] minvalue | NO MINVALUE ] [ MAXVALUE [=] maxvalue | NO MAXVALUE ] [ START [ WITH | = ] start ] [ CACHE [=] cache ] [ [ NO ] CYCLE ] ENGINE=xxx COMMENT=".." SELECT NEXT VALUE FOR sequence_name; SELECT NEXTVAL(sequence_name); SELECT PREVIOUS VALUE FOR sequence_name; SELECT LASTVAL(sequence_name); SHOW CREATE SEQUENCE sequence_name; SHOW CREATE TABLE sequence_name; CREATE TABLE sequence-structure ... SEQUENCE=1 ALTER TABLE sequence RENAME TO sequence2; RENAME TABLE sequence RENAME TO sequence2; DROP [TEMPORARY] SEQUENCE sequence_name [IF EXISTS] See https://jira.mariadb.org/browse/MDEV-10139 for progress. See commit messages in bb-10.2-sequence for what is still to be done. Some documentation can be found at: https://mariadb.com/kb/en/mariadb/create-sequence/ This will be improved as the work progress. One of the goals with the SEQUENCE implementation is that all old tools, like mysqldump, should work unchanged, while still keeping normal usage of sequence standard compatibly. The make this possible, the sequence is currently implemented as a table with a few exclusive properties. The main disadvantage of having sequence as a table is that it uses the same name space as tables. In other words, you can't have a table and a sequence of the same name. The benefit is that sequences shows up in 'show tables', one can create a sequence also with 'create table' and drop it with 'drop table'. One can select from it as from any other table. This ensures that all old tools that works with tables should work with sequences. The special properties for sequence tables are: - A sequence table has always one row. - When one creates a sequence, either with CREATE TABLE or CREATE SEQUENCE, one row will be inserted. - Normal table options works for CREATE SEQUENCE. One can use ENGINE=xxx, COMMENT=xxx etc. - If one tries to insert into a sequence table, the single row will be updated. This allows mysqldump to work but also gives the additional benefit that one can change all properties of a sequence with a single insert. New applications can of course also use ALTER SEQUENCE. - Updates to the sequence table will change the single row. - Doing a select on the sequence shows the current state of the sequence, except the values that are reserved in the cache. The column 'next_value' shows the next value not reserved by the cache. - Truncate on a sequence table will give an error. - Alter table and rename works on sequences. - If one creates a sequence with INCREMENT 0, then the sequence will use auto_increment_increment and auto_increment_offset for the sequence, just like AUTO_INCREMENT. This allows sequences to work reliable in a master-master environment and with Galera. Internally sequence tables are created as a normal table without rollback (InnoDB, Aria and MySAM supports this) with is wrapped by a sequence engine. This allowed me to create sequences with almost no performance impact for normal tables. (The cost is one 'if' per insert if binary log is enabled). MariaDB 10.3 will support both the ANSI SQL and Oracle syntax for creating and accessing sequences. As ANSI SQL doesn't have an easy access to the last generated value, 103 also supports 'PREVIOUS VALUE FOR sequence_name', like IBM DB2, and LASTVAL(sequence_name) as PostgreSQL. I want to thank Jianwe Zhao from Aliyun for his work on SEQUENCE in AliSQL which gave me ideas and inspiration for this work. I also want to thank Marko Mäkelä for his help in the InnoDB part of the code. There is still a lot of work to fix edge cases, but in the current implementation most major things seams to work... Comments, suggestions or questions? Regards, Monty

2 1

Re: [Maria-developers] 5e94fe1: MDEV-15754 Server crashes in fill_record_n_invoke_before_triggers upon
by Sergei Golubchik 17 Apr '18

17 Apr '18

Hi, Sachin! Looks good, thanks! Ok to push. On Apr 16, sachin wrote: > revision-id: 5e94fe189d6325028613b2c830082e07ff303acb (mariadb-10.3.5-156-g5e94fe1) > parent(s): f0e4f94c230326a2f2e608e4119530d775c37b21 > author: Sachin Setiya > committer: Sachin Setiya > timestamp: 2018-04-16 16:27:11 +0530 > message: > > MDEV-15754 Server crashes in fill_record_n_invoke_before_triggers upon ... > insert into table with TIMESTAMP INVISIBLE > > Problem:- The segfault occurs because value is null but since timestamp field > is VISIBLE it expects a value , and it tries to call value->save_in_field(.. > Timestamp field should not be visible this is the problem. > > Solution:- While we clone field for record0_field we don't honor the field > _visibility , this patch changes that. > > diff --git a/sql/sql_trigger.cc b/sql/sql_trigger.cc > index 4c1e2a5..d523779 100644 > --- a/sql/sql_trigger.cc > +++ b/sql/sql_trigger.cc > @@ -1240,6 +1240,7 @@ bool Table_triggers_list::prepare_record_accessors(TABLE *table) > return 1; > > f->flags= (*fld)->flags; > + f->invisible= (*fld)->invisible; > f->null_ptr= null_ptr; > f->null_bit= null_bit; > if (null_bit == 128) Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] Questions about AS OF (possibly bugs found)
by Alexander Barkov 16 Apr '18

16 Apr '18

Hi Sergei, I'm looking at this use rule in sql_yacc.yy: history_point: temporal_literal { $$= Vers_history_point(VERS_TIMESTAMP, $1); } | function_call_keyword_timestamp { $$= Vers_history_point(VERS_TIMESTAMP, $1); } | opt_history_unit simple_expr { $$= Vers_history_point($1, $2); } ; AS OF DATE 'xxx' - returns Vers_history_point(VERS_TIMESTAMP) AS OF DATE('xxx') - returns Vers_history_point(VERS_UNDEFINED) AS OF TIMESTAMP('2001-01-01') - returns Vers_history_point(VERS_TIMESTAMP) AS OF COALESCE(TIMESTAMP('2001-01-01')) - returns Vers_history_point(VERS_UNDEFINED) Perhaps this is not expected. Also, this code looks suspicious: void Vers_history_point::resolve_unit(bool timestamps_only) { if (item && unit == VERS_UNDEFINED) { if (item->type() == Item::FIELD_ITEM || timestamps_only) unit= VERS_TIMESTAMP; else if (item->result_type() == INT_RESULT || item->result_type() == REAL_RESULT) unit= VERS_TRX_ID; else unit= VERS_TIMESTAMP; } } Why DECIMAL_RESULT is not handled? Why only Item::FIELD_ITEM is checked? Can't "history_point" be simplified to something like: history_point: simple_expr { $$= Vers_history_point(VERS_UNDEFINED, $1); } | TRANSACTION_SYM simple_expr { $$= Vers_history_point(VERS_TRX_ID, $2); } ; I'm saying "something like" because TRANSACTION_SYM will cause a conflict with simple_expr. So probably it should be cought somehow else, by catching Item_ident and checking it name. Btw, an SP variable with name "TRANSACTION" does not work well: This script works fine: DELIMITER $$ BEGIN NOT ATOMIC DECLARE TRANSACTION int DEFAULT 10; SELECT TRANSACTION; END; $$ DELIMITER ; This script: DELIMITER $$ BEGIN NOT ATOMIC DECLARE TRANSACTION int DEFAULT 10; SELECT * FROM t1 FOR SYSTEM_TIME AS OF TRANSACTION; END; $$ DELIMITER ; returns an error: ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '; Note, if I rename "TRANSACTION" to "TRANSACTION1", it works fine. Thanks!

3 8

[Maria-developers] innobase/sync0policy.h converts opaque pthread_t to ULINT
by Teodor Mircea Ionita 16 Apr '18

16 Apr '18

Hi, While trying to build on macOS 10.13.4 with clang 9.1.0 I originally came across this error: -- .../storage/innobase/include/sync0policy.h:79:4: error: cannot initialize a parameter of type 'int64' (aka 'long long') with an rvalue of type 'os_thread_id_t' (aka '_opaque_pthread_t *') my_atomic_storelint(&m_thread_id, os_thread_get_curr_id()); --- This problem was originally reported in MDEV-15778. Since pthread_t on macOS is a pointer to a struct named _opaque_pthread_t, the "obvious" solution for this was to explicitly cast to ULINT which in turn is defined as size_t, fine so far. Solution was tested in PR#691. However, upon testing my patch with buildbot, on slave kvm-fulltest2, which is 64bit qemu vm with Ubuntu precise-i386 (32bit) running and gcc compiling in -march=i686. The above "fix" now yields: -- /home/buildbot/buildbot/build/mariadb-10.3.6/storage/innobase/include/sync0policy.h:79:4: error: invalid cast from type ‘os_thread_id_t {aka long unsigned int}’ to type ‘ulint {aka unsigned int}’ --- Now reconsidering more carefully, this looks more serious then initially thought, posix pthread_t(3) is defined as: -- POSIX.1 allows an implementation wide freedom in choosing the type used to represent a thread ID; for example, representation using either an arithmetic type or a structure is permitted. Therefore, variables of type pthread_t can't portably be compared using the C equality operator (==); use pthread_equal(3)instead. --- So in this particular platform spec, where pthread_t on Linux is defined as unsigned long int integral type and if LP64 model is used, this implies pthread_t always being 64bit in either both 32 and 64 arch and trying to convert to size_t on a 32bit arch will fail. Now given that the standard particularly states pthread_t as being opaque and no assumptions should be made in regards to the actual type, not even an initializer type (which ULINT_UNDEFINED defines as -1), trying to cast m_thread_id to any other type ( (ULINT) ) either implicitly or explicit is not portable. Is this a bug that needs fixing or just impossible to handle pthread id atomically in any other fashion? How about a test case I can run to confirm/infirm erratic behavior? Since I don't have access to the particular bb slave instance to check with sizeof() maybe somebody else can help me here to confirm this? Relevant links bellow: https://jira.mariadb.org/browse/MDEV-15778 https://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/12317/… https://linux.die.net/man/3/pthread_self https://github.com/MariaDB/server/pull/691 Cheers, Teodor

4 5

Re: [Maria-developers] a91fdd1: Fixed the bug mdev-15575 different results when using CTE and big_tables=1.
by Sergei Golubchik 16 Apr '18

16 Apr '18

Hi, Igor On Apr 14, IgorBabaev wrote: > revision-id: a91fdd1288b3cbc5616f2ef9e0ce04bf7f0ef9ce (mariadb-10.2.14-35-ga91fdd1) > parent(s): 740fc2ae084f8f81990de557d696aefbc507752d > author: Igor Babaev > committer: Igor Babaev > timestamp: 2018-04-14 21:32:41 -0700 > message: > > Fixed the bug mdev-15575 different results when using CTE and big_tables=1. please, make the first line MDEV-15575 different results when using CTE and big_tables=1 most other developers use that pattern, so your commits really stand out in the history, e.g.: 4c7a1a1 MDEV-15780 : mariabackup does not handle absolute names in for system tablespaces 9124590 MDEV-12903: encryption.innodb_encryption_discard_import fails in buildbot with FOUND vs NOT FOUND dd12779 MDEV-15832 With innodb_fast_shutdown=3, skip the rollback of connected transactions 740fc2a Fixed mdev-15765 BETWEEN not working in certain cases 8334ace MDEV-14380 Reduce possibility to timing-induced error in test and the reason for that is that many tools show only first 50 characters of the commit summary (first line), so starting it with the "Fixed the bug " does not add new information, but only pushes the relevant content to the right to be truncated. > This bug happened due to a defect of the implementation of the handler > function ha_delete_all_rows() for the ARIA engine. > The function maria_delete_all_rows() truncated the table, but it didn't > touch the write cache, so the cache's write offset was not reset. > In the scenario like in the function st_select_lex_unit::exec_recursive > when first all records were deleted from the table and then several new > records were added some metadata became inconsistent with the state of > the cache. As a result the table scan function could not read records > at the end of the table. > The same defect could be found in the implementation of ha_delete_all_rows() > for the MYISAM engine mi_delete_all_rows(). This is ok to push > Additionally made late instantiation for the temporary table used to store > rows that were used for each new iteration when executing a recursive CTE. I didn't review this part. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] Questions regarding closing partitions in MDEV-11084
by Mattias Jonsson 16 Apr '18

16 Apr '18

Hi MariaDB Devs, I tried to evaluate spider engine and found an issue where it crashes, most likely due to MDEV-11084 (Stacktrace and reproducible test case attached). That also leads me to wonder about the performance for partitioned tables after MDEV-11084, when it seems to close partitions whenever it is not used in a statement (i.e. require it to be reopened in the next statement that would use another partition, effectively not using the open table cache). I cannot see anything mentioned in the final commit message hinting on that it closes partitions not used in the current query, but in the previous patches it was mentioned without any reason. Why does it close the already opened partitions? https://github.com/MariaDB/server/blob/10.3/sql/ha_partition.cc#L8365 I would not mind opening the partitions only when they are to be used (even though there are engines that need to be tested more), but closing them makes no sense to me performance wise. Also notice that the partitions first will be put back into the open table cache and then on the next query the non-used partitions will be closed and the needed ones be (re)-opened. Regards Mattias Jonsson

1 0

Re: [Maria-developers] Fwd: [Commits] b32031991e4: initial oracle parser fix
by Alexander Barkov 16 Apr '18

16 Apr '18

Hello Sanja, I reviewed your recent changes in "10.3-MDEV-11953" (and the attached additional patch for sql_yacc_ora.yy) I have some proposals: 1. Can you please move huge pieces of the code from sql_yacc.yy to LEX or other relevant classes? It makes the grammar much more readable (patches are aslo much more readable). I'd move the relevant pieces of the code to LEX as a separate patch, even before fixing the grammar. 2. You're adding too many main_select_push() and pop_select(). Please move them to upper level rules (it should be possible in many cases). Add new helper rules when needed. For example, this piece of code repeats many times: + { + if (Lex->main_select_push()) + MYSQL_YYABORT; + } + expr + { + Lex->pop_select(); //main select + $$= $3; It deserved a rule, say, expr_with_select_push_pop. You can find a better name :) - Serg and I spent a lot of time working on this task: MDEV-8909 union parser cleanup (and its 13 dependency tasks, and 3 related tasks, see the "Issue links" section in MDEV). We think that it should be the parser who disallows bad grammar, instead of post-analysis with raising errors like ER_CANT_USE_OPTION_HERE. Please keep using the same approach. Thanks! On 04/06/2018 07:37 PM, Oleksandr Byelkin wrote: > > > > -------- Weitergeleitete Nachricht -------- > Betreff: [Commits] b32031991e4: initial oracle parser fix > Datum: Fri, 06 Apr 2018 17:34:31 +0200 > Von: Oleksandr Byelkin <sanja(a)mariadb.com> > Antwort an: maria-developers(a)lists.launchpad.net > An: commits(a)mariadb.org > > > > revision-id: b32031991e4ba270f5c221f7f6068732d09efd53 > (mariadb-10.3.5-110-gb32031991e4) > parent(s): 60a502eb7efbe87bd8e3af3e3ece49a9eb83f8b3 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-04-06 17:05:34 +0200 > message: > > initial oracle parser fix > > --- > sql/sql_yacc.yy | 8 +- > sql/sql_yacc_ora.yy | 2085 > +++++++++++++++++++++++++++++---------------------- > 2 files changed, 1183 insertions(+), 910 deletions(-) > > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index e4cbff7114e..9372269b05d 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -904,7 +904,7 @@ bool my_yyoverflow(short **a, YYSTYPE **b, size_t > *yystacksize); > %parse-param { THD *thd } > %lex-param { THD *thd } > /* > - Currently there are 96 shift/reduce conflicts. > + Currently there are 135 shift/reduce conflicts. > We should not introduce new conflicts any more. > */ > %expect 135 > @@ -9244,7 +9244,7 @@ table_value_constructor: > lex->push_select(sel)) > MYSQL_YYABORT; > sel->init_select(); > - sel->braces= FALSE; > + sel->braces= FALSE; // just initialisation > } > values_list > { > @@ -13763,8 +13763,10 @@ single_multi: > } > | table_wild_list > { > + /* XXX > if (Lex->main_select_push()) > MYSQL_YYABORT; > + */ > mysql_init_multi_delete(Lex); > YYPS->m_lock_type= TL_READ_DEFAULT; > YYPS->m_mdl_type= MDL_SHARED_READ; > @@ -13779,8 +13781,10 @@ single_multi: > } > | FROM table_alias_ref_list > { > + /* XXX > if (Lex->main_select_push()) > MYSQL_YYABORT; > + */ > mysql_init_multi_delete(Lex); > YYPS->m_lock_type= TL_READ_DEFAULT; > YYPS->m_mdl_type= MDL_SHARED_READ; > diff --git a/sql/sql_yacc_ora.yy b/sql/sql_yacc_ora.yy > index c1512434d72..4a3ee348f89 100644 > --- a/sql/sql_yacc_ora.yy > +++ b/sql/sql_yacc_ora.yy > @@ -186,6 +186,20 @@ void ORAerror(THD *thd, const char *s) > LEX_CSTRING name; > uint offset; > } sp_cursor_name_and_offset; > + struct > + { > + enum sub_select_type unit_type; > + bool distinct; > + } unit_operation; > + struct > + { > + SELECT_LEX *first; > + SELECT_LEX *prev_last; > + } select_list; > + SQL_I_List<ORDER> *select_order; > + Lex_select_lock select_lock; > + Lex_select_limit select_limit; > + Lex_order_limit_lock *order_limit_lock; > > /* pointers */ > Create_field *create_field; > @@ -231,6 +245,7 @@ void ORAerror(THD *thd, const char *s) > > handlerton *db_type; > st_select_lex *select_lex; > + st_select_lex_unit *select_lex_unit; > struct p_elem_val *p_elem_value; > class Window_frame *window_frame; > class Window_frame_bound *window_frame_bound; > @@ -240,7 +255,6 @@ void ORAerror(THD *thd, const char *s) > /* enums */ > enum enum_sp_suid_behaviour sp_suid; > enum enum_view_suid view_suid; > - enum sub_select_type unit_type; > enum Condition_information_item::Name cond_info_item_name; > enum enum_diag_condition_item_name diag_condition_item_name; > enum Diagnostics_information::Which_area diag_area; > @@ -277,10 +291,10 @@ bool my_yyoverflow(short **a, YYSTYPE **b, size_t > *yystacksize); > %parse-param { THD *thd } > %lex-param { THD *thd } > /* > - Currently there are 104 shift/reduce conflicts. > + Currently there are 99 shift/reduce conflicts. > We should not introduce new conflicts any more. > */ > -%expect 104 > +%expect 99 > > /* > Comments for TOKENS. > @@ -599,6 +613,9 @@ bool my_yyoverflow(short **a, YYSTYPE **b, size_t > *yystacksize); > %token LEAVES > %token LEAVE_SYM > %token LEFT /* SQL-2003-R */ > +%token LEFT_PAREN_ALT /* INTERNAL */ > +%token LEFT_PAREN_WITH /* INTERNAL */ > +%token LEFT_PAREN_LIKE /* INTERNAL */ > %token LESS_SYM > %token LEVEL_SYM > %token LEX_HOSTNAME > @@ -1063,7 +1080,8 @@ bool my_yyoverflow(short **a, YYSTYPE **b, size_t > *yystacksize); > NCHAR_STRING > > %type <lex_str_ptr> > - opt_table_alias > + opt_table_alias_clause > + table_alias_clause > > %type <lex_string_with_pos> > ident ident_with_tok_start > @@ -1111,7 +1129,7 @@ bool my_yyoverflow(short **a, YYSTYPE **b, size_t > *yystacksize); > opt_temporary all_or_any opt_distinct opt_glimit_clause > opt_ignore_leaves fulltext_options union_option > opt_not > - select_derived_init transaction_access_mode_types > + transaction_access_mode_types > opt_natural_language_mode opt_query_expansion > opt_ev_status opt_ev_on_completion ev_on_completion opt_ev_comment > ev_alter_on_schedule_completion opt_ev_rename_to opt_ev_sql_stmt > @@ -1231,9 +1249,8 @@ bool my_yyoverflow(short **a, YYSTYPE **b, size_t > *yystacksize); > join_table_list join_table > table_factor table_ref esc_table_ref > table_primary_ident table_primary_derived > - select_derived derived_table_list > - select_derived_union > - derived_query_specification > + derived_table_list table_reference_list_parens > + nested_table_reference_list join_table_parens > %type <date_time_type> date_time_type; > %type <interval> interval > > @@ -1276,12 +1293,16 @@ bool my_yyoverflow(short **a, YYSTYPE **b, > size_t *yystacksize); > UNDERSCORE_CHARSET > > %type <select_lex> subselect > - get_select_lex get_select_lex_derived > query_specification > - query_term_union_not_ready > - query_term_union_ready > + table_value_constructor > + simple_table > + query_primary > + query_primary_parens > + > +%type <select_lex_unit> > query_expression_body > - select_paren_derived > + query_expression > + query_expression_unit > > %type <boolfunc2creator> comp_op > > @@ -1293,7 +1314,21 @@ bool my_yyoverflow(short **a, YYSTYPE **b, size_t > *yystacksize); > > %type <virtual_column> opt_check_constraint check_constraint > virtual_column_func > column_default_expr > -%type <unit_type> unit_type_decl > + > +%type <unit_operation> unit_type_decl > + > +%type <select_lock> > + opt_select_lock_type > + select_lock_type > + opt_lock_wait_timeout_new > + > +%type <select_limit> opt_limit_clause limit_clause limit_options > + > +%type <order_limit_lock> > + query_expression_tail > + order_or_limit > + > +%type <select_order> opt_order_clause order_clause order_list > > %type <NONE> > analyze_stmt_command > @@ -1313,7 +1348,7 @@ bool my_yyoverflow(short **a, YYSTYPE **b, size_t > *yystacksize); > assign_to_keycache_parts > preload_list preload_list_or_parts preload_keys preload_keys_parts > select_item_list select_item values_list no_braces > - opt_limit_clause delete_limit_clause fields opt_values values > + delete_limit_clause fields opt_values values > procedure_list procedure_list2 procedure_item > field_def handler opt_generated_always > opt_ignore opt_column opt_restrict > @@ -1333,9 +1368,8 @@ bool my_yyoverflow(short **a, YYSTYPE **b, size_t > *yystacksize); > table_to_table_list table_to_table opt_table_list opt_as > handler_rkey_function handler_read_or_scan > single_multi table_wild_list table_wild_one opt_wild > - union_clause union_list > - subselect_start opt_and charset > - subselect_end select_var_list select_var_list_init help > + opt_and charset > + select_var_list select_var_list_init help > opt_extended_describe shutdown > opt_format_json > prepare prepare_src execute deallocate > @@ -1606,14 +1640,22 @@ deallocate_or_drop: > ; > > prepare: > - PREPARE_SYM ident FROM prepare_src > + PREPARE_SYM > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > + ident FROM prepare_src > { > LEX *lex= thd->lex; > if (lex->table_or_sp_used()) > my_yyabort_error((ER_SUBQUERIES_NOT_SUPPORTED, MYF(0), > "PREPARE..FROM")); > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > lex->sql_command= SQLCOM_PREPARE; > - lex->prepared_stmt_name= $2; > + lex->prepared_stmt_name= $3; > + Lex->pop_select(); //main select > } > ; > > @@ -1632,10 +1674,21 @@ execute: > LEX *lex= thd->lex; > lex->sql_command= SQLCOM_EXECUTE; > lex->prepared_stmt_name= $2; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > execute_using > - {} > - | EXECUTE_SYM IMMEDIATE_SYM prepare_src > + { > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + } > + | EXECUTE_SYM IMMEDIATE_SYM > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > + prepare_src > { > if (Lex->table_or_sp_used()) > my_yyabort_error((ER_SUBQUERIES_NOT_SUPPORTED, MYF(0), > @@ -1643,7 +1696,11 @@ execute: > Lex->sql_command= SQLCOM_EXECUTE_IMMEDIATE; > } > execute_using > - {} > + { > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + } > ; > > execute_using: > @@ -1928,13 +1985,20 @@ connection_name: > /* create a table */ > > create: > - create_or_replace opt_temporary TABLE_SYM opt_if_not_exists > table_ident > + create_or_replace opt_temporary TABLE_SYM opt_if_not_exists > { > LEX *lex= thd->lex; > lex->create_info.init(); > + if (lex->main_select_push()) > + MYSQL_YYABORT; > + lex->current_select->parsing_place= BEFORE_OPT_LIST; > if (lex->set_command_with_check(SQLCOM_CREATE_TABLE, $2, $1 > | $4)) > MYSQL_YYABORT; > - if (!lex->builtin_select.add_table_to_list(thd, $5, NULL, > + } > + table_ident > + { > + LEX *lex= thd->lex; > + if (!lex->builtin_select.add_table_to_list(thd, $6, NULL, > TL_OPTION_UPDATING, > TL_WRITE, > MDL_EXCLUSIVE)) > MYSQL_YYABORT; > @@ -1960,13 +2024,16 @@ create: > ER_WARN_USING_OTHER_HANDLER, > ER_THD(thd, > ER_WARN_USING_OTHER_HANDLER), > > hton_name(lex->create_info.db_type)->str, > - $5->table.str); > + $6->table.str); > } > create_table_set_open_action_and_adjust_tables(lex); > + Lex->pop_select(); //main select > } > | create_or_replace opt_temporary SEQUENCE_SYM opt_if_not_exists > table_ident > { > LEX *lex= thd->lex; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > lex->create_info.init(); > if (lex->set_command_with_check(SQLCOM_CREATE_SEQUENCE, $2, > $1 | $4)) > MYSQL_YYABORT; > @@ -2021,42 +2088,69 @@ create: > $5->table.str); > } > create_table_set_open_action_and_adjust_tables(lex); > + Lex->pop_select(); //main select > } > - | create_or_replace opt_unique INDEX_SYM opt_if_not_exists ident > + | create_or_replace opt_unique INDEX_SYM opt_if_not_exists > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > + ident > opt_key_algorithm_clause > ON table_ident > { > - if (Lex->add_create_index_prepare($8)) > + if (Lex->add_create_index_prepare($9)) > MYSQL_YYABORT; > - if (Lex->add_create_index($2, &$5, $6, $1 | $4)) > + if (Lex->add_create_index($2, &$6, $7, $1 | $4)) > MYSQL_YYABORT; > } > '(' key_list ')' opt_lock_wait_timeout normal_key_options > - opt_index_lock_algorithm { } > - | create_or_replace fulltext INDEX_SYM opt_if_not_exists ident > + opt_index_lock_algorithm > + { > + Lex->pop_select(); //main select > + } > + | create_or_replace fulltext INDEX_SYM > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > + opt_if_not_exists ident > ON table_ident > { > - if (Lex->add_create_index_prepare($7)) > + if (Lex->add_create_index_prepare($8)) > MYSQL_YYABORT; > - if (Lex->add_create_index($2, &$5, HA_KEY_ALG_UNDEF, $1 | $4)) > + if (Lex->add_create_index($2, &$6, HA_KEY_ALG_UNDEF, $1 | $5)) > MYSQL_YYABORT; > } > '(' key_list ')' opt_lock_wait_timeout fulltext_key_options > - opt_index_lock_algorithm { } > - | create_or_replace spatial INDEX_SYM opt_if_not_exists ident > + opt_index_lock_algorithm > + { > + Lex->pop_select(); //main select > + } > + | create_or_replace spatial INDEX_SYM > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > + opt_if_not_exists ident > ON table_ident > { > - if (Lex->add_create_index_prepare($7)) > + if (Lex->add_create_index_prepare($8)) > MYSQL_YYABORT; > - if (Lex->add_create_index($2, &$5, HA_KEY_ALG_UNDEF, $1 | $4)) > + if (Lex->add_create_index($2, &$6, HA_KEY_ALG_UNDEF, $1 | $5)) > MYSQL_YYABORT; > } > '(' key_list ')' opt_lock_wait_timeout spatial_key_options > - opt_index_lock_algorithm { } > + opt_index_lock_algorithm > + { > + Lex->pop_select(); //main select > + } > | create_or_replace DATABASE opt_if_not_exists ident > { > Lex->create_info.default_table_charset= NULL; > Lex->create_info.used_fields= 0; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > opt_create_database_options > { > @@ -2064,51 +2158,94 @@ create: > if (lex->set_command_with_check(SQLCOM_CREATE_DB, 0, $1 | $3)) > MYSQL_YYABORT; > lex->name= $4; > + Lex->pop_select(); //main select > } > | create_or_replace definer_opt opt_view_suid VIEW_SYM > opt_if_not_exists table_ident > { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > if (Lex->add_create_view(thd, $1 | $5, > DTYPE_ALGORITHM_UNDEFINED, $3, $6)) > MYSQL_YYABORT; > } > view_list_opt AS view_select > - { } > + { > + Lex->pop_select(); //main select > + } > | create_or_replace view_algorithm definer_opt opt_view_suid > VIEW_SYM > opt_if_not_exists table_ident > { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > if (Lex->add_create_view(thd, $1 | $6, $2, $4, $7)) > MYSQL_YYABORT; > } > view_list_opt AS view_select > - { } > + { > + Lex->pop_select(); //main select > + } > | create_or_replace definer_opt TRIGGER_SYM > - { Lex->create_info.set($1); } > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + Lex->create_info.set($1); > + } > trigger_tail > - { } > + { > + Lex->pop_select(); //main select > + } > | create_or_replace definer_opt PROCEDURE_SYM > - { Lex->create_info.set($1); } > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + Lex->create_info.set($1); > + } > sp_tail_standalone > - { } > + { > + Lex->pop_select(); //main select > + } > | create_or_replace definer_opt EVENT_SYM > - { Lex->create_info.set($1); } > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + Lex->create_info.set($1); > + } > event_tail > - { } > + { > + Lex->pop_select(); //main select > + } > | create_or_replace definer FUNCTION_SYM > - { Lex->create_info.set($1); } > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + Lex->create_info.set($1); > + } > sf_tail_standalone > - { } > + { > + Lex->pop_select(); //main select > + } > | create_or_replace no_definer FUNCTION_SYM > - { Lex->create_info.set($1); } > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + Lex->create_info.set($1); > + } > create_function_tail > - { } > + { > + Lex->pop_select(); //main select > + } > | create_or_replace no_definer AGGREGATE_SYM FUNCTION_SYM > { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > Lex->create_info.set($1); > Lex->udf.type= UDFTYPE_AGGREGATE; > } > udf_tail > - { } > + { > + Lex->pop_select(); //main select > + } > | create_or_replace USER_SYM opt_if_not_exists clear_privileges > grant_list > opt_require_clause opt_resource_options > { > @@ -2815,8 +2952,13 @@ call: > { > if (Lex->call_statement_start(thd, $2)) > MYSQL_YYABORT; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > + opt_sp_cparam_list > + { > + Lex->pop_select(); //main select > } > - opt_sp_cparam_list {} > ; > > /* CALL parameters */ > @@ -3346,10 +3488,16 @@ raise_stmt: > ; > > signal_stmt: > - SIGNAL_SYM signal_value opt_set_signal_information > + SIGNAL_SYM > { > - if (Lex->add_signal_statement(thd, $2)) > + if (Lex->main_select_push()) > + YYABORT; > + } > + signal_value opt_set_signal_information > + { > + if (Lex->add_signal_statement(thd, $3)) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > } > ; > > @@ -3475,9 +3623,14 @@ resignal_stmt: > ; > > get_diagnostics: > - GET_SYM which_area DIAGNOSTICS_SYM diagnostics_information > + GET_SYM which_area DIAGNOSTICS_SYM > + { > + if (Lex->main_select_push()) > + YYABORT; > + } > + diagnostics_information > { > - Diagnostics_information *info= $4; > + Diagnostics_information *info= $5; > > info->set_which_da($2); > > @@ -3486,6 +3639,7 @@ get_diagnostics: > > if (Lex->m_sql_cmd == NULL) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > } > ; > > @@ -3666,8 +3820,26 @@ sp_decl_idents: > > sp_opt_default: > /* Empty */ { $$ = NULL; } > - | DEFAULT expr { $$ = $2; } > - | SET_VAR expr { $$ = $2; } > + | DEFAULT > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > + expr > + { > + Lex->pop_select(); //main select > + $$ = $3; > + } > + | SET_VAR > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > + expr > + { > + Lex->pop_select(); //main select > + $$ = $3; > + } > ; > > sp_proc_stmt: > @@ -3784,10 +3956,15 @@ sp_proc_stmt_statement: > > sp_proc_stmt_return: > RETURN_SYM > - { Lex->sphead->reset_lex(thd); } > + { > + Lex->sphead->reset_lex(thd); > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > expr > { > LEX *lex= Lex; > + lex->pop_select(); //main select > sp_head *sp= lex->sphead; > if (sp->m_handler->add_instr_freturn(thd, sp, lex->spcont, > $3, lex) || > @@ -3902,6 +4079,8 @@ assignment_source_expr: > { > DBUG_ASSERT(thd->free_list == NULL); > Lex->sphead->reset_lex(thd, $1); > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > expr > { > @@ -3910,6 +4089,7 @@ assignment_source_expr: > $$->sp_lex_in_use= true; > $$->set_item_and_free_list($3, thd->free_list); > thd->free_list= NULL; > + Lex->pop_select(); //main select > if ($$->sphead->restore_lex(thd)) > MYSQL_YYABORT; > } > @@ -4030,9 +4210,14 @@ sp_fetch_list: > ; > > sp_if: > - { Lex->sphead->reset_lex(thd); } > + { > + Lex->sphead->reset_lex(thd); > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > expr THEN_SYM > { > + Lex->pop_select(); //main select > LEX *lex= Lex; > sp_head *sp= lex->sphead; > sp_pcontext *ctx= lex->spcont; > @@ -4144,12 +4329,18 @@ case_stmt_specification: > ; > > case_stmt_body: > - { Lex->sphead->reset_lex(thd); /* For expr $2 */ } > + { > + Lex->sphead->reset_lex(thd); /* For expr $2 */ > + > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > expr > { > if (Lex->case_stmt_action_expr($2)) > MYSQL_YYABORT; > > + Lex->pop_select(); //main select > if (Lex->sphead->restore_lex(thd)) > MYSQL_YYABORT; > } > @@ -4173,6 +4364,9 @@ simple_when_clause: > WHEN_SYM > { > Lex->sphead->reset_lex(thd); /* For expr $3 */ > + > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > expr > { > @@ -4181,6 +4375,8 @@ simple_when_clause: > LEX *lex= Lex; > if (lex->case_stmt_action_when($3, true)) > MYSQL_YYABORT; > + > + lex->pop_select(); //main select > /* For expr $3 */ > if (lex->sphead->restore_lex(thd)) > MYSQL_YYABORT; > @@ -4197,12 +4393,17 @@ searched_when_clause: > WHEN_SYM > { > Lex->sphead->reset_lex(thd); /* For expr $3 */ > + > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > expr > { > LEX *lex= Lex; > if (lex->case_stmt_action_when($3, false)) > MYSQL_YYABORT; > + > + lex->pop_select(); //main select > /* For expr $3 */ > if (lex->sphead->restore_lex(thd)) > MYSQL_YYABORT; > @@ -4442,6 +4643,7 @@ while_body: > LEX *lex= Lex; > if (lex->sp_while_loop_expression(thd, $1)) > MYSQL_YYABORT; > + Lex->pop_select(); //main select pushed before while_body use > if (lex->sphead->restore_lex(thd)) > MYSQL_YYABORT; > } > @@ -4454,7 +4656,12 @@ while_body: > > repeat_body: > sp_proc_stmts1 UNTIL_SYM > - { Lex->sphead->reset_lex(thd); } > + { > + Lex->sphead->reset_lex(thd); > + > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > expr END REPEAT_SYM > { > LEX *lex= Lex; > @@ -4465,6 +4672,8 @@ repeat_body: > if (i == NULL || > lex->sphead->add_instr(i)) > MYSQL_YYABORT; > + > + lex->pop_select(); //main select > if (lex->sphead->restore_lex(thd)) > MYSQL_YYABORT; > /* We can shortcut the cont_backpatch here */ > @@ -4493,8 +4702,12 @@ sp_labeled_control: > if (Lex->sp_push_loop_label(thd, &$1)) > MYSQL_YYABORT; > Lex->sphead->reset_lex(thd); > + > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > while_body pop_sp_loop_label > + > { } > | labels_declaration_oracle FOR_SYM > { > @@ -4546,9 +4759,13 @@ sp_unlabeled_control: > if (Lex->sp_push_loop_empty_label(thd)) > MYSQL_YYABORT; > Lex->sphead->reset_lex(thd); > + > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > while_body > { > + // while body pop main select > Lex->sp_pop_loop_empty_label(thd); > } > | FOR_SYM > @@ -4981,20 +5198,10 @@ size_number: > */ > > create_body: > - '(' create_field_list ')' > + create_field_list_parens > { Lex->create_info.option_list= NULL; } > opt_create_table_options opt_create_partitioning > opt_create_select {} > | opt_create_table_options opt_create_partitioning > opt_create_select {} > - /* > - the following rule is redundant, but there's a shift/reduce > - conflict that prevents the rule above from parsing a syntax like > - CREATE TABLE t1 (SELECT 1); > - */ > - | '(' create_select_query_specification ')' > - | '(' create_select_query_specification ')' > - { Select->set_braces(1);} union_list {} > - | '(' create_select_query_specification ')' > - { Select->set_braces(1);} union_order_or_limit {} > | create_like > { > > @@ -5010,7 +5217,7 @@ create_body: > > create_like: > LIKE table_ident { $$= $2; } > - | '(' LIKE table_ident ')' { $$= $3; } > + | LEFT_PAREN_LIKE LIKE table_ident ')' { $$= $3; } > ; > > opt_create_select: > @@ -5019,23 +5226,42 @@ opt_create_select: > ; > > create_select_query_expression: > - opt_with_clause SELECT_SYM create_select_part2 > opt_table_expression > - create_select_part4 > - { > - Select->set_braces(0); > - Select->set_with_clause($1); > + query_expression > + { > + SELECT_LEX *first_select= $1->first_select(); > + > + if (Lex->sql_command == SQLCOM_INSERT || > + Lex->sql_command == SQLCOM_REPLACE) > + { > + if (Lex->sql_command == SQLCOM_INSERT) > + Lex->sql_command= SQLCOM_INSERT_SELECT; > + else > + Lex->sql_command= SQLCOM_REPLACE_SELECT; > + } > + Lex->insert_select_hack(first_select); > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + > } > - union_clause > - | opt_with_clause SELECT_SYM create_select_part2 > - create_select_part3_union_not_ready create_select_part4 > + | LEFT_PAREN_WITH with_clause query_expression_body ')' > { > - Select->set_with_clause($1); > + SELECT_LEX *first_select= $3->first_select(); > + $3->set_with_clause($2); > + $2->attach_to(first_select); > + > + Lex->insert_select_hack(first_select); > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + > + if (Lex->sql_command == SQLCOM_INSERT || > + Lex->sql_command == SQLCOM_REPLACE) > + { > + if (Lex->sql_command == SQLCOM_INSERT) > + Lex->sql_command= SQLCOM_INSERT_SELECT; > + else > + Lex->sql_command= SQLCOM_REPLACE_SELECT; > + } > } > - | '(' create_select_query_specification ')' > - | '(' create_select_query_specification ')' > - { Select->set_braces(1);} union_list {} > - | '(' create_select_query_specification ')' > - { Select->set_braces(1);} union_order_or_limit {} > ; > > opt_create_partitioning: > @@ -5126,8 +5352,13 @@ partition_entry: > We enter here when opening the frm file to translate > partition info string into part_info data structure. > */ > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > + partition > + { > + Lex->pop_select(); //main select > } > - partition {} > ; > > partition: > @@ -5742,56 +5973,6 @@ opt_part_option: > End of partition parser part > */ > > -create_select_query_specification: > - opt_with_clause SELECT_SYM create_select_part2 > create_select_part3 > - create_select_part4 > - { > - Select->set_with_clause($1); > - } > - ; > - > -create_select_part2: > - { > - LEX *lex=Lex; > - if (lex->sql_command == SQLCOM_INSERT) > - lex->sql_command= SQLCOM_INSERT_SELECT; > - else if (lex->sql_command == SQLCOM_REPLACE) > - lex->sql_command= SQLCOM_REPLACE_SELECT; > - /* > - The following work only with the local list, the global list > - is created correctly in this case > - */ > - > lex->current_select->table_list.save_and_clear(&lex->save_list); > - mysql_init_select(lex); > - lex->current_select->parsing_place= SELECT_LIST; > - } > - select_options select_item_list > - { > - Select->parsing_place= NO_MATTER; > - } > - ; > - > -create_select_part3: > - opt_table_expression > - | create_select_part3_union_not_ready > - ; > - > -create_select_part3_union_not_ready: > - table_expression order_or_limit > - | order_or_limit > - ; > - > -create_select_part4: > - opt_select_lock_type > - { > - /* > - The following work only with the local list, the global list > - is created correctly in this case > - */ > - Lex->current_select->table_list.push_front(&Lex->save_list); > - } > - ; > - > opt_as: > /* empty */ {} > | AS {} > @@ -6195,6 +6376,13 @@ create_field_list: > } > ; > > +create_field_list_parens: > + LEFT_PAREN_ALT field_list ')' > + { > + Lex->create_last_non_select_table= Lex->last_table(); > + } > + ; > + > field_list: > field_list_item > | field_list ',' field_list_item > @@ -6462,6 +6650,8 @@ parse_vcol_expr: > Prevent the end user from invoking this command. > */ > MYSQL_YYABORT_UNLESS(Lex->parse_vcol_expr); > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > expr > { > @@ -6469,13 +6659,29 @@ parse_vcol_expr: > if (!v) > MYSQL_YYABORT; > Lex->last_field->vcol_info= v; > + Lex->pop_select(); //main select > } > ; > > parenthesized_expr: > - subselect > + remember_tok_start > + query_expression > { > - $$= new (thd->mem_root) Item_singlerow_subselect(thd, $1); > + if (!Lex->expr_allows_subselect || > + Lex->sql_command == (int)SQLCOM_PURGE) > + { > + thd->parse_error(ER_SYNTAX_ERROR, $1); > + MYSQL_YYABORT; > + } > + > + // Add the subtree of subquery to the current SELECT_LEX > + SELECT_LEX *curr_sel= Lex->select_stack_head(); > + DBUG_ASSERT(Lex->current_select == curr_sel); > + curr_sel->register_unit($2, &curr_sel->context); > + curr_sel->add_statistics($2); > + > + $$= new (thd->mem_root) > + Item_singlerow_subselect(thd, $2->first_select()); > if ($$ == NULL) > MYSQL_YYABORT; > } > @@ -7474,6 +7680,8 @@ alter: > Lex->alter_info.reset(); > Lex->no_write_to_binlog= 0; > Lex->create_info.storage_media= HA_SM_DEFAULT; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > DBUG_ASSERT(!Lex->m_sql_cmd); > } > alter_options TABLE_SYM table_ident opt_lock_wait_timeout > @@ -7494,12 +7702,15 @@ alter: > Lex->m_sql_cmd= new (thd->mem_root) Sql_cmd_alter_table(); > if (Lex->m_sql_cmd == NULL) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > } > } > | ALTER DATABASE ident_or_empty > { > Lex->create_info.default_table_charset= NULL; > Lex->create_info.used_fields= 0; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > create_database_options > { > @@ -7508,6 +7719,7 @@ alter: > lex->name= $3; > if (lex->name.str == NULL && lex->copy_db_to(&lex->name)) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > } > | ALTER DATABASE ident UPGRADE_SYM DATA_SYM DIRECTORY_SYM NAME_SYM > { > @@ -7523,6 +7735,8 @@ alter: > > if (lex->sphead) > my_yyabort_error((ER_SP_NO_DROP_SP, MYF(0), "PROCEDURE")); > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > lex->sp_chistics.init(); > } > sp_a_chistics > @@ -7531,6 +7745,9 @@ alter: > > lex->sql_command= SQLCOM_ALTER_PROCEDURE; > lex->spname= $3; > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > } > | ALTER FUNCTION_SYM sp_name > { > @@ -7538,6 +7755,8 @@ alter: > > if (lex->sphead) > my_yyabort_error((ER_SP_NO_DROP_SP, MYF(0), "FUNCTION")); > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > lex->sp_chistics.init(); > } > sp_a_chistics > @@ -7546,14 +7765,23 @@ alter: > > lex->sql_command= SQLCOM_ALTER_FUNCTION; > lex->spname= $3; > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > } > | ALTER view_algorithm definer_opt opt_view_suid VIEW_SYM > table_ident > { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > if (Lex->add_alter_view(thd, $2, $4, $6)) > MYSQL_YYABORT; > } > view_list_opt AS view_select > - {} > + { > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + } > | ALTER definer_opt opt_view_suid VIEW_SYM table_ident > /* > We have two separate rules for ALTER VIEW rather that > @@ -7561,14 +7789,22 @@ alter: > with the ALTER EVENT below. > */ > { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > if (Lex->add_alter_view(thd, VIEW_ALGORITHM_INHERIT, $3, $5)) > MYSQL_YYABORT; > } > view_list_opt AS view_select > - {} > + { > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + } > | ALTER definer_opt remember_name EVENT_SYM sp_name > { > - /* > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + /* > It is safe to use Lex->spname because > ALTER EVENT xxx RENATE TO yyy DO ALTER EVENT RENAME TO > is not allowed. Lex->spname is used in the case of RENAME TO > @@ -7600,6 +7836,8 @@ alter: > */ > Lex->sql_command= SQLCOM_ALTER_EVENT; > Lex->stmt_definition_end= (char*)YYLIP->get_cpp_ptr(); > + > + Lex->pop_select(); //main select > } > | ALTER TABLESPACE alter_tablespace_info > { > @@ -7643,6 +7881,8 @@ alter: > lex->create_info.init(); > lex->no_write_to_binlog= 0; > DBUG_ASSERT(!lex->m_sql_cmd); > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > table_ident > { > @@ -7660,6 +7900,9 @@ alter: > Lex->m_sql_cmd= new (thd->mem_root) > Sql_cmd_alter_sequence($3); > if (Lex->m_sql_cmd == NULL) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > } > ; > > @@ -8336,9 +8579,13 @@ checksum: > lex->sql_command = SQLCOM_CHECKSUM; > /* Will be overridden during execution. */ > YYPS->m_lock_type= TL_UNLOCK; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > table_list opt_checksum_type > - {} > + { > + Lex->pop_select(); //main select > + } > ; > > opt_checksum_type: > @@ -8364,6 +8611,8 @@ repair: > lex->alter_info.reset(); > /* Will be overridden during execution. */ > YYPS->m_lock_type= TL_UNLOCK; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > repair_table_or_view > { > @@ -8372,6 +8621,7 @@ repair: > lex->m_sql_cmd= new (thd->mem_root) Sql_cmd_repair_table(); > if (lex->m_sql_cmd == NULL) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > } > ; > > @@ -8400,6 +8650,8 @@ analyze: > ANALYZE_SYM opt_no_write_to_binlog table_or_tables > { > LEX *lex=Lex; > + if (lex->main_select_push()) > + YYABORT; > lex->sql_command = SQLCOM_ANALYZE; > lex->no_write_to_binlog= $2; > lex->check_opt.init(); > @@ -8414,6 +8666,7 @@ analyze: > lex->m_sql_cmd= new (thd->mem_root) Sql_cmd_analyze_table(); > if (lex->m_sql_cmd == NULL) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > } > ; > > @@ -8530,6 +8783,8 @@ check: CHECK_SYM > lex->alter_info.reset(); > /* Will be overridden during execution. */ > YYPS->m_lock_type= TL_UNLOCK; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > check_view_or_table > { > @@ -8540,6 +8795,7 @@ check: CHECK_SYM > lex->m_sql_cmd= new (thd->mem_root) Sql_cmd_check_table(); > if (lex->m_sql_cmd == NULL) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > } > ; > > @@ -8577,6 +8833,8 @@ optimize: > lex->alter_info.reset(); > /* Will be overridden during execution. */ > YYPS->m_lock_type= TL_UNLOCK; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > table_list opt_lock_wait_timeout > { > @@ -8585,6 +8843,7 @@ optimize: > lex->m_sql_cmd= new (thd->mem_root) Sql_cmd_optimize_table(); > if (lex->m_sql_cmd == NULL) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > } > ; > > @@ -8598,9 +8857,13 @@ rename: > RENAME table_or_tables > { > Lex->sql_command= SQLCOM_RENAME_TABLE; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > table_to_table_list > - {} > + { > + Lex->pop_select(); //main select > + } > | RENAME USER_SYM clear_privileges rename_list > { > Lex->sql_command = SQLCOM_RENAME_USER; > @@ -8694,10 +8957,14 @@ preload: > LEX *lex=Lex; > lex->sql_command=SQLCOM_PRELOAD_KEYS; > lex->alter_info.reset(); > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > preload_list_or_parts > - {} > - ; > + { > + Lex->pop_select(); //main select > + } > + ; > > preload_list_or_parts: > preload_keys_parts > @@ -8763,192 +9030,353 @@ opt_ignore_leaves: > > > select: > - opt_with_clause select_init > + query_expression > { > - LEX *lex= Lex; > - lex->sql_command= SQLCOM_SELECT; > - lex->current_select->set_with_clause($1); > + Lex->selects_allow_into= TRUE; > + Lex->selects_allow_procedure= TRUE; > + Lex->set_main_unit($1); > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + > + SELECT_LEX *fake= Lex->unit.fake_select_lex; > + if (fake) > + { > + fake->no_table_names_allowed= 1; > + Lex->push_select(fake); > + } > + else > + Lex->push_select(Lex->first_select_lex()); > + Lex->sql_command= SQLCOM_SELECT; > } > ; > > -select_init: > - SELECT_SYM select_options_and_item_list select_init3 > - | '(' select_paren ')' > - | '(' select_paren ')' union_list > - | '(' select_paren ')' union_order_or_limit > - ; > > -union_list_part2: > - SELECT_SYM select_options_and_item_list > select_init3_union_query_term > - | '(' select_paren_union_query_term ')' > - | '(' select_paren_union_query_term ')' union_list > - | '(' select_paren_union_query_term ')' union_order_or_limit > +simple_table: > + query_specification { $$= $1; } > + | table_value_constructor { $$= $1; } > ; > - > -select_paren: > + > +table_value_constructor: > + VALUES > + { > + LEX *lex= Lex; > + SELECT_LEX *sel; > + //lex->field_list.empty(); > + lex->many_values.empty(); > + lex->insert_list=0; > + if (!(sel= lex->alloc_select(TRUE)) || > + lex->push_select(sel)) > + MYSQL_YYABORT; > + sel->init_select(); > + sel->braces= FALSE; // just initialisation > + } > + values_list > + { > + LEX *lex=Lex; > + $$= lex->pop_select(); // above TVC select > + if (!($$->tvc= > + new (lex->thd->mem_root) > table_value_constr(lex->many_values, > + $$, > + > $$->options))) > + MYSQL_YYABORT; > + lex->many_values.empty(); > + } > + ; > + > +query_specification: > + SELECT_SYM > { > - /* > - In order to correctly parse UNION's global ORDER BY we > need to > - set braces before parsing the clause. > - */ > - Lex->current_select->set_braces(true); > + SELECT_LEX *sel; > + LEX *lex= Lex; > + if (!(sel= lex->alloc_select(TRUE)) || > + lex->push_select(sel)) > + MYSQL_YYABORT; > + sel->init_select(); > + sel->braces= FALSE; > } > - SELECT_SYM select_options_and_item_list select_part3 > - opt_select_lock_type > + select_options > { > - DBUG_ASSERT(Lex->current_select->braces); > + Select->parsing_place= SELECT_LIST; > } > - | '(' select_paren ')' > - ; > - > -select_paren_union_query_term: > + select_item_list > { > - /* > - In order to correctly parse UNION's global ORDER BY we > need to > - set braces before parsing the clause. > - */ > - Lex->current_select->set_braces(true); > + Select->parsing_place= NO_MATTER; > } > - SELECT_SYM select_options_and_item_list > select_part3_union_query_term > - opt_select_lock_type > + opt_into > + opt_from_clause > + opt_where_clause > + opt_group_clause > + opt_having_clause > + opt_window_clause > { > - DBUG_ASSERT(Lex->current_select->braces); > + $$= Lex->pop_select(); > } > - | '(' select_paren_union_query_term ')' > ; > > -select_paren_view: > - { > - /* > - In order to correctly parse UNION's global ORDER BY we > need to > - set braces before parsing the clause. > - */ > - Lex->current_select->set_braces(true); > - } > - SELECT_SYM select_options_and_item_list select_part3_view > - opt_select_lock_type > - { > - DBUG_ASSERT(Lex->current_select->braces); > - } > - | '(' select_paren_view ')' > +opt_from_clause: > + /* Empty */ > + | from_clause > + ; > + > +query_primary: > + simple_table > + { $$= $1; } > + | query_primary_parens > + { $$= $1; } > ; > > -/* The equivalent of select_paren for nested queries. */ > -select_paren_derived: > +query_primary_parens: > + '(' query_expression_unit > { > - Lex->current_select->set_braces(true); > + SELECT_LEX *last= $2->pre_last_parse->next_select(); > + int cmp= > cmp_unit_op($2->first_select()->next_select()->linkage, > + last->linkage); > + if (cmp < 0) > + { > + if (!check_intersect_prefix($2->first_select())) > + { > + if (Lex->pop_new_select_and_wrap() == NULL) > + MYSQL_YYABORT; > + } > + } > + Lex->push_select($2->fake_select_lex); > } > - SELECT_SYM select_part2_derived > - opt_table_expression > - opt_order_clause > - opt_limit_clause > - opt_select_lock_type > + query_expression_tail ')' > { > - DBUG_ASSERT(Lex->current_select->braces); > - $$= Lex->current_select->master_unit()->first_select(); > + Lex->pop_select(); > + if ($4) > + { > + ($4)->set_to($2->fake_select_lex); > + } > + $$= $2->first_select(); > } > - | '(' select_paren_derived ')' { $$= $2; } > - ; > - > -select_init3: > - opt_table_expression > - opt_select_lock_type > + | '(' query_primary > { > - /* Parentheses carry no meaning here */ > - Lex->current_select->set_braces(false); > + Lex->push_select($2); > } > - union_clause > - | select_part3_union_not_ready > - opt_select_lock_type > + query_expression_tail ')' > { > - /* Parentheses carry no meaning here */ > - Lex->current_select->set_braces(false); > + Lex->pop_select(); > + $$= $2; > + $$->braces= TRUE; > + if ($4) > + { > + if ($2->next_select()) > + { > + SELECT_LEX_UNIT *unit= $2->master_unit(); > + if (!unit) > + unit= Lex->create_unit($2); > + if (!unit) > + YYABORT; > + if (!unit->fake_select_lex->is_set_query_expr_tail) > + $4->set_to(unit->fake_select_lex); > + else > + { > + $$= Lex->wrap_unit_into_derived(unit); > + if (!$$) > + YYABORT; > + $4->set_to($$); > + } > + } > + else if (!$2->is_set_query_expr_tail) > + { > + $4->set_to($2); > + } > + else > + { > + SELECT_LEX_UNIT *unit= Lex->create_unit($2); > + if (!unit) > + YYABORT; > + $$= Lex->wrap_unit_into_derived(unit); > + if (!$$) > + YYABORT; > + $4->set_to($$); > + } > + } > } > ; > > - > -select_init3_union_query_term: > - opt_table_expression > - opt_select_lock_type > +query_expression_unit: > + query_primary > + unit_type_decl > + query_primary > { > - /* Parentheses carry no meaning here */ > - Lex->current_select->set_braces(false); > + SELECT_LEX *sel1; > + SELECT_LEX *sel2; > + if (!$1->next_select()) > + sel1= $1; > + else > + { > + sel1= Lex->wrap_unit_into_derived($1->master_unit()); > + if (!sel1) > + YYABORT; > + } > + if (!$3->next_select()) > + sel2= $3; > + else > + { > + sel2= Lex->wrap_unit_into_derived($3->master_unit()); > + if (!sel2) > + YYABORT; > + } > + sel1->link_neighbour(sel2); > + sel2->set_linkage_and_distinct($2.unit_type, $2.distinct); > + $$= Lex->create_unit(sel1); > + $$->pre_last_parse= sel1; > + if ($$ == NULL) > + YYABORT; > } > - union_clause > - | select_part3_union_not_ready_noproc > - opt_select_lock_type > + | query_expression_unit > + unit_type_decl > + query_primary > { > - /* Parentheses carry no meaning here */ > - Lex->current_select->set_braces(false); > + SELECT_LEX *sel1; > + if (!$3->next_select()) > + sel1= $3; > + else > + { > + sel1= Lex->wrap_unit_into_derived($3->master_unit()); > + if (!sel1) > + YYABORT; > + } > + SELECT_LEX *last= $1->pre_last_parse->next_select(); > + > + int cmp= cmp_unit_op($2.unit_type, last->linkage); > + if (cmp == 0) > + { > + // do nothing, this part will be just connected > + } > + else if (cmp > 0) > + { > + // Store beginning and continue to connect parts > + if (Lex->push_new_select($1->pre_last_parse)) > + MYSQL_YYABORT; > + } > + else /* cmp < 0 */ > + { > + // wrap stored part in a select, then continue to connect > parts > + if (!check_intersect_prefix($1->first_select())) > + { > + if ((last= Lex->pop_new_select_and_wrap()) == NULL) > + MYSQL_YYABORT; > + last->set_master_unit($1); > + } > + } > + last->link_neighbour(sel1); > + sel1->set_master_unit($1); > + sel1->set_linkage_and_distinct($2.unit_type, $2.distinct); > + $$= $1; > + $$->pre_last_parse= last; > } > ; > > - > -select_init3_view: > - opt_table_expression opt_select_lock_type > +query_expression_body: > + query_primary > { > - Lex->current_select->set_braces(false); > + Lex->push_select($1); > } > - | opt_table_expression opt_select_lock_type > + query_expression_tail > { > - Lex->current_select->set_braces(false); > + Lex->pop_select(); > + SELECT_LEX *sel= $1; > + if ($3) > + { > + if ($1->next_select()) > + { > + SELECT_LEX_UNIT *unit= $1->master_unit(); > + if (!unit) > + unit= Lex->create_unit($1); > + if (!unit) > + YYABORT; > + if (!unit->fake_select_lex->is_set_query_expr_tail) > + $3->set_to(unit->fake_select_lex); > + else > + { > + SELECT_LEX *sel= Lex->wrap_unit_into_derived(unit); > + if (!sel) > + YYABORT; > + $3->set_to(sel); > + } > + } > + else if (!$1->is_set_query_expr_tail) > + $3->set_to($1); > + else > + { > + SELECT_LEX_UNIT *unit= $1->master_unit(); > + if (!unit) > + unit= Lex->create_unit($1); > + if (!unit) > + YYABORT; > + sel= Lex->wrap_unit_into_derived(unit); > + if (!sel) > + YYABORT; > + $3->set_to(sel); > + } > + } > + $$= Lex->create_unit(sel); > + if ($$ == NULL) > + YYABORT; > } > - union_list_view > - | order_or_limit opt_select_lock_type > + | query_expression_unit > { > - Lex->current_select->set_braces(false); > + SELECT_LEX *last= $1->pre_last_parse->next_select(); > + int cmp= > cmp_unit_op($1->first_select()->next_select()->linkage, > + last->linkage); > + if (cmp < 0) > + { > + if (!check_intersect_prefix($1->first_select())) > + { > + if (Lex->pop_new_select_and_wrap() == NULL) > + MYSQL_YYABORT; > + } > + } > + Lex->push_select($1->fake_select_lex); > } > - | table_expression order_or_limit opt_select_lock_type > + query_expression_tail > { > - Lex->current_select->set_braces(false); > + Lex->pop_select(); > + if ($3) > + { > + ($3)->set_to($1->fake_select_lex); > + } > + $$= $1; > } > ; > > -/* > - The SELECT parts after select_item_list that cannot be followed by > UNION. > -*/ > - > -select_part3: > - opt_table_expression > - | select_part3_union_not_ready > - ; > - > -select_part3_union_query_term: > - opt_table_expression > - | select_part3_union_not_ready_noproc > - ; > - > -select_part3_view: > - opt_table_expression > - | order_or_limit > - | table_expression order_or_limit > +query_expression: > + opt_with_clause > + query_expression_body > + { > + if ($1) > + { > + $2->set_with_clause($1); > + $1->attach_to($2->first_select()); > + } > + $$= $2; > + } > ; > > -select_part3_union_not_ready: > - select_part3_union_not_ready_noproc > - | table_expression procedure_clause > - | table_expression order_or_limit procedure_clause > - ; > +subselect: > + remember_tok_start > + query_expression > + { > + if (!Lex->expr_allows_subselect || > + Lex->sql_command == (int)SQLCOM_PURGE) > + { > + thd->parse_error(ER_SYNTAX_ERROR, $1); > + MYSQL_YYABORT; > + } > > -select_part3_union_not_ready_noproc: > - order_or_limit > - | into opt_table_expression opt_order_clause opt_limit_clause > - | table_expression into > - | table_expression order_or_limit > - | table_expression order_or_limit into > - ; > + // Add the subtree of subquery to the current SELECT_LEX > + SELECT_LEX *curr_sel= Lex->select_stack_head(); > + DBUG_ASSERT(Lex->current_select == curr_sel); > + if (curr_sel) > + { > + curr_sel->register_unit($2, &curr_sel->context); > + curr_sel->add_statistics($2); > + } > > -select_options_and_item_list: > - { > - LEX *lex= Lex; > - SELECT_LEX *sel= lex->current_select; > - if (sel->linkage != UNION_TYPE) > - mysql_init_select(lex); > - lex->current_select->parsing_place= SELECT_LIST; > - } > - select_options select_item_list > - { > - Select->parsing_place= NO_MATTER; > + $$= $2->first_select(); > } > ; > > @@ -8956,18 +9384,6 @@ select_options_and_item_list: > /** > <table expression>, as in the SQL standard. > */ > -table_expression: > - from_clause > - opt_where_clause > - opt_group_clause > - opt_having_clause > - opt_window_clause > - ; > - > -opt_table_expression: > - /* Empty */ > - | table_expression > - ; > > from_clause: > FROM table_reference_list > @@ -9005,59 +9421,63 @@ select_option: > query_expression_option > | SQL_NO_CACHE_SYM > { > - /* > - Allow this flag only on the first top-level SELECT > statement, if > - SQL_CACHE wasn't specified, and only once per query. > - */ > - if (Lex->current_select != &Lex->builtin_select) > - my_yyabort_error((ER_CANT_USE_OPTION_HERE, MYF(0), > "SQL_NO_CACHE")); > - if (Lex->builtin_select.sql_cache == SELECT_LEX::SQL_CACHE) > - my_yyabort_error((ER_WRONG_USAGE, MYF(0), "SQL_CACHE", > "SQL_NO_CACHE")); > - if (Lex->builtin_select.sql_cache == SELECT_LEX::SQL_NO_CACHE) > + /* > + Allow this flag once per query. > + */ > + if (Select->options & OPTION_NO_QUERY_CACHE) > my_yyabort_error((ER_DUP_ARGUMENT, MYF(0), "SQL_NO_CACHE")); > - > - Lex->safe_to_cache_query=0; > - Lex->builtin_select.options&= ~OPTION_TO_QUERY_CACHE; > - Lex->builtin_select.sql_cache= SELECT_LEX::SQL_NO_CACHE; > + Select->options|= OPTION_NO_QUERY_CACHE; > } > | SQL_CACHE_SYM > { > - /* > - Allow this flag only on the first top-level SELECT > statement, if > - SQL_NO_CACHE wasn't specified, and only once per query. > - */ > - if (Lex->current_select != &Lex->builtin_select) > - my_yyabort_error((ER_CANT_USE_OPTION_HERE, MYF(0), > "SQL_CACHE")); > - if (Lex->builtin_select.sql_cache == SELECT_LEX::SQL_NO_CACHE) > - my_yyabort_error((ER_WRONG_USAGE, MYF(0), "SQL_NO_CACHE", > "SQL_CACHE")); > - if (Lex->builtin_select.sql_cache == SELECT_LEX::SQL_CACHE) > + /* > + Allow this flag once per query. > + */ > + if (Select->options & OPTION_TO_QUERY_CACHE) > my_yyabort_error((ER_DUP_ARGUMENT, MYF(0), "SQL_CACHE")); > - > - Lex->safe_to_cache_query=1; > - Lex->builtin_select.options|= OPTION_TO_QUERY_CACHE; > - Lex->builtin_select.sql_cache= SELECT_LEX::SQL_CACHE; > + Select->options|= OPTION_TO_QUERY_CACHE; > } > ; > > opt_select_lock_type: > /* empty */ > - | FOR_SYM UPDATE_SYM opt_lock_wait_timeout > + { $$.empty(); } > + | select_lock_type > + { $$= $1; } > + ; > + > +select_lock_type: > + FOR_SYM UPDATE_SYM opt_lock_wait_timeout_new > { > - LEX *lex=Lex; > - lex->current_select->lock_type= TL_WRITE; > - lex->current_select->set_lock_for_tables(TL_WRITE); > - lex->safe_to_cache_query=0; > + $$= $3; > + $$.defined_lock= TRUE; > + $$.update_lock= TRUE; > } > - | LOCK_SYM IN_SYM SHARE_SYM MODE_SYM opt_lock_wait_timeout > + | LOCK_SYM IN_SYM SHARE_SYM MODE_SYM opt_lock_wait_timeout_new > { > - LEX *lex=Lex; > - lex->current_select->lock_type= TL_READ_WITH_SHARED_LOCKS; > - lex->current_select-> > - set_lock_for_tables(TL_READ_WITH_SHARED_LOCKS); > - lex->safe_to_cache_query=0; > + $$= $5; > + $$.defined_lock= TRUE; > + $$.update_lock= FALSE; > } > ; > > +opt_lock_wait_timeout_new: > + /* empty */ > + { > + $$.empty(); > + } > + | WAIT_SYM ulong_num > + { > + $$.defined_timeout= TRUE; > + $$.timeout= $2; > + } > + | NOWAIT_SYM > + { > + $$.defined_timeout= TRUE; > + $$.timeout= 0; > + } > + ; > + > select_item_list: > select_item_list ',' select_item > | select_item > @@ -11361,10 +11781,15 @@ esc_table_ref: > /* Equivalent to <table reference list> in the SQL:2003 standard. */ > /* Warning - may return NULL in case of incomplete SELECT */ > derived_table_list: > - esc_table_ref { $$=$1; } > + esc_table_ref > + { > + $$=$1; > + Select->add_joined_table($1); > + } > | derived_table_list ',' esc_table_ref > { > MYSQL_YYABORT_UNLESS($1 && ($$=$3)); > + Select->add_joined_table($3); > } > ; > > @@ -11383,11 +11808,18 @@ join_table: > left-associative joins. > */ > table_ref normal_join table_ref %prec TABLE_REF_PRIORITY > - { MYSQL_YYABORT_UNLESS($1 && ($$=$3)); $3->straight=$2; } > + { > + MYSQL_YYABORT_UNLESS($1 && ($$=$3)); > + Select->add_joined_table($1); > + Select->add_joined_table($3); > + $3->straight=$2; > + } > | table_ref normal_join table_ref > ON > { > MYSQL_YYABORT_UNLESS($1 && $3); > + Select->add_joined_table($1); > + Select->add_joined_table($3); > /* Change the current name resolution context to a local > context. */ > if (push_new_name_resolution_context(thd, $1, $3)) > MYSQL_YYABORT; > @@ -11404,6 +11836,8 @@ join_table: > USING > { > MYSQL_YYABORT_UNLESS($1 && $3); > + Select->add_joined_table($1); > + Select->add_joined_table($3); > } > '(' using_list ')' > { > @@ -11414,6 +11848,8 @@ join_table: > | table_ref NATURAL inner_join table_factor > { > MYSQL_YYABORT_UNLESS($1 && ($$=$4)); > + Select->add_joined_table($1); > + Select->add_joined_table($4); > $4->straight=$3; > add_join_natural($1,$4,NULL,Select); > } > @@ -11423,6 +11859,8 @@ join_table: > ON > { > MYSQL_YYABORT_UNLESS($1 && $5); > + Select->add_joined_table($1); > + Select->add_joined_table($5); > /* Change the current name resolution context to a local > context. */ > if (push_new_name_resolution_context(thd, $1, $5)) > MYSQL_YYABORT; > @@ -11439,6 +11877,8 @@ join_table: > | table_ref LEFT opt_outer JOIN_SYM table_factor > { > MYSQL_YYABORT_UNLESS($1 && $5); > + Select->add_joined_table($1); > + Select->add_joined_table($5); > } > USING '(' using_list ')' > { > @@ -11449,6 +11889,8 @@ join_table: > | table_ref NATURAL LEFT opt_outer JOIN_SYM table_factor > { > MYSQL_YYABORT_UNLESS($1 && $6); > + Select->add_joined_table($1); > + Select->add_joined_table($6); > add_join_natural($1,$6,NULL,Select); > $6->outer_join|=JOIN_TYPE_LEFT; > $$=$6; > @@ -11459,6 +11901,8 @@ join_table: > ON > { > MYSQL_YYABORT_UNLESS($1 && $5); > + Select->add_joined_table($1); > + Select->add_joined_table($5); > /* Change the current name resolution context to a local > context. */ > if (push_new_name_resolution_context(thd, $1, $5)) > MYSQL_YYABORT; > @@ -11476,6 +11920,8 @@ join_table: > | table_ref RIGHT opt_outer JOIN_SYM table_factor > { > MYSQL_YYABORT_UNLESS($1 && $5); > + Select->add_joined_table($1); > + Select->add_joined_table($5); > } > USING '(' using_list ')' > { > @@ -11487,6 +11933,8 @@ join_table: > | table_ref NATURAL RIGHT opt_outer JOIN_SYM table_factor > { > MYSQL_YYABORT_UNLESS($1 && $6); > + Select->add_joined_table($1); > + Select->add_joined_table($6); > add_join_natural($6,$1,NULL,Select); > LEX *lex= Lex; > if (!($$= lex->current_select->convert_right_join())) > @@ -11521,40 +11969,70 @@ use_partition: > $$= $3; > } > ; > - > -/* > - This is a flattening of the rules <table factor> and <table primary> > - in the SQL:2003 standard, since we don't have <sample clause> > > - I.e. > - <table factor> ::= <table primary> [ <sample clause> ] > -*/ > -/* Warning - may return NULL in case of incomplete SELECT */ > table_factor: > - table_primary_ident > - | table_primary_derived > + table_primary_ident { $$= $1; } > + | table_primary_derived { $$= $1; } > + | join_table_parens { $$= $1; } > + | table_reference_list_parens { $$= $1; } > ; > > +table_reference_list_parens: > + '(' table_reference_list_parens ')' { $$= $2; } > + | '(' nested_table_reference_list ')' > + { > + if (!($$= Select->end_nested_join(thd))) > + MYSQL_YYABORT; > + } > + ; > + > +nested_table_reference_list: > + table_ref ',' table_ref > + { > + if (Select->init_nested_join(thd)) > + MYSQL_YYABORT; > + Select->add_joined_table($1); > + Select->add_joined_table($3); > + $$= $1->embedding; > + } > + | nested_table_reference_list ',' table_ref > + { > + Select->add_joined_table($3); > + $$= $1; > + } > + ; > + > +join_table_parens: > + '(' join_table_parens ')' { $$= $2; } > + | '(' join_table ')' > + { > + LEX *lex= Lex; > + if (!($$= lex->current_select->nest_last_join(thd))) > + { > + thd->parse_error(); > + MYSQL_YYABORT; > + } > + } > + ; > + > + > table_primary_ident: > + table_ident opt_use_partition > + opt_table_alias_clause opt_key_definition > { > SELECT_LEX *sel= Select; > sel->table_join_options= 0; > - } > - table_ident opt_use_partition opt_table_alias opt_key_definition > - { > - if (!($$= Select->add_table_to_list(thd, $2, $4, > + if (!($$= Select->add_table_to_list(thd, $1, $3, > > Select->get_table_join_options(), > YYPS->m_lock_type, > YYPS->m_mdl_type, > Select->pop_index_hints(), > - $3))) > + $2))) > MYSQL_YYABORT; > - Select->add_joined_table($$); > } > ; > > > - > /* > Represents a flattening of the following rules from the SQL:2003 > standard. This sub-rule corresponds to the sub-rule > @@ -11572,242 +12050,56 @@ table_primary_ident: > */ > > table_primary_derived: > - '(' get_select_lex select_derived_union ')' opt_table_alias > + query_primary_parens table_alias_clause > { > - /* Use $2 instead of Lex->current_select as derived table will > - alter value of Lex->current_select. */ > - if (!($3 || $5) && $2->embedding && > - !$2->embedding->nested_join->join_list.elements) > + LEX *lex=Lex; > + lex->derived_tables|= DERIVED_SUBQUERY; > + $1->linkage= DERIVED_TABLE_TYPE; > + $1->braces= FALSE; > + // Add the subtree of subquery to the current SELECT_LEX > + SELECT_LEX *curr_sel= Lex->select_stack_head(); > + DBUG_ASSERT(Lex->current_select == curr_sel); > + SELECT_LEX_UNIT *unit= $1->master_unit(); > + if (!unit) > { > - /* we have a derived table ($3 == NULL) but no alias, > - Since we are nested in further parentheses so we > - can pass NULL to the outer level parentheses > - Permits parsing of "((((select ...))) as xyz)" */ > - $$= 0; > + unit= Lex->create_unit($1); > + if (!unit) > + YYABORT; > } > - else if (!$3) > - { > - /* Handle case of derived table, alias may be NULL if there > - are no outer parentheses, add_table_to_list() will throw > - error in this case */ > - LEX *lex=Lex; > - lex->check_automatic_up(UNSPECIFIED_TYPE); > - SELECT_LEX *sel= lex->current_select; > - SELECT_LEX_UNIT *unit= sel->master_unit(); > - lex->current_select= sel= unit->outer_select(); > - Table_ident *ti= new (thd->mem_root) Table_ident(unit); > - if (ti == NULL) > - MYSQL_YYABORT; > - if (!($$= sel->add_table_to_list(thd, > - ti, $5, 0, > - TL_READ, MDL_SHARED_READ))) > + curr_sel->register_unit(unit, &curr_sel->context); > + curr_sel->add_statistics(unit); > > - MYSQL_YYABORT; > - sel->add_joined_table($$); > - //lex->pop_context("derived"); > - lex->nest_level--; > - } > - else if ($5 != NULL) > - { > - /* > - Tables with or without joins within parentheses cannot > - have aliases, and we ruled out derived tables above. > - */ > - thd->parse_error(); > - MYSQL_YYABORT; > - } > - else > - { > - /* nested join: FROM (t1 JOIN t2 ...), > - nest_level is the same as in the outer query */ > - $$= $3; > - } > - /* > - Fields in derived table can be used in upper select in > - case of merge. We do not add HAVING fields because we do > - not merge such derived. We do not add union because > - also do not merge them > - */ > - if ($$ && $$->derived && > - !$$->derived->first_select()->next_select()) > - > $$->select_lex->add_where_field($$->derived->first_select()); > - } > - /* Represents derived table with WITH clause */ > - | '(' get_select_lex subselect_start > - with_clause query_expression_body > - subselect_end ')' opt_table_alias > - { > - LEX *lex=Lex; > - SELECT_LEX *sel= $2; > - SELECT_LEX_UNIT *unit= $5->master_unit(); > Table_ident *ti= new (thd->mem_root) Table_ident(unit); > if (ti == NULL) > MYSQL_YYABORT; > - $5->set_with_clause($4); > - lex->current_select= sel; > - if (!($$= sel->add_table_to_list(lex->thd, > - ti, $8, 0, > - TL_READ, MDL_SHARED_READ))) > + if (!($$= curr_sel->add_table_to_list(lex->thd, > + ti, $2, 0, > + TL_READ, > MDL_SHARED_READ))) > MYSQL_YYABORT; > - sel->add_joined_table($$); > - } > - ; > - > -/* > - This rule accepts just about anything. The reason is that we have > - empty-producing rules in the beginning of rules, in this case > - subselect_start. This forces bison to take a decision which rules to > - reduce by long before it has seen any tokens. This approach ties us > - to a very limited class of parseable languages, and unfortunately > - SQL is not one of them. The chosen 'solution' was this rule, which > - produces just about anything, even complete bogus statements, for > - instance ( table UNION SELECT 1 ). > - Fortunately, we know that the semantic value returned by > - select_derived is NULL if it contained a derived table, and a pointer to > - the base table's TABLE_LIST if it was a base table. So in the rule > - regarding union's, we throw a parse error manually and pretend it > - was bison that did it. > - > - Also worth noting is that this rule concerns query expressions in > - the from clause only. Top level select statements and other types of > - subqueries have their own union rules. > -*/ > -select_derived_union: > - select_derived > - | select_derived union_order_or_limit > - { > - if ($1) > - { > - thd->parse_error(); > - MYSQL_YYABORT; > - } > - } > - | select_derived union_head_non_top > - { > - if ($1) > - { > - thd->parse_error(); > - MYSQL_YYABORT; > - } > } > - union_list_derived_part2 > - | derived_query_specification opt_select_lock_type > - | derived_query_specification order_or_limit opt_select_lock_type > - | derived_query_specification opt_select_lock_type > union_list_derived > - ; > - > -union_list_derived_part2: > - query_term_union_not_ready { Lex->pop_context(); } > - | query_term_union_ready { Lex->pop_context(); } > - | query_term_union_ready { Lex->pop_context(); } > union_list_derived > - ; > - > -union_list_derived: > - union_head_non_top union_list_derived_part2 > - ; > - > - > -/* The equivalent of select_init2 for nested queries. */ > -select_init2_derived: > - select_part2_derived > - { > - Select->set_braces(0); > - } > - ; > - > -/* The equivalent of select_part2 for nested queries. */ > -select_part2_derived: > - { > - LEX *lex= Lex; > - SELECT_LEX *sel= lex->current_select; > - if (sel->linkage != UNION_TYPE) > - mysql_init_select(lex); > - lex->current_select->parsing_place= SELECT_LIST; > - } > - opt_query_expression_options select_item_list > - { > - Select->parsing_place= NO_MATTER; > - } > - ; > - > -/* handle contents of parentheses in join expression */ > -select_derived: > - get_select_lex_derived derived_table_list > + | '(' > + query_expression > + ')' table_alias_clause > { > - LEX *lex= Lex; > - /* for normal joins, $2 != NULL and end_nested_join() != NULL, > - for derived tables, both must equal NULL */ > + LEX *lex=Lex; > + lex->derived_tables|= DERIVED_SUBQUERY; > + $2->first_select()->linkage= DERIVED_TABLE_TYPE; > > - if (!($$= $1->end_nested_join(lex->thd)) && $2) > - MYSQL_YYABORT; > - if (!$2 && $$) > - { > - thd->parse_error(); > - MYSQL_YYABORT; > - } > - } > - ; > > -/* > - Similar to query_specification, but for derived tables. > - Example: the inner parenthesized SELECT in this query: > - SELECT * FROM (SELECT * FROM t1); > -*/ > -derived_query_specification: > - SELECT_SYM select_derived_init select_derived2 > - { > - if ($2) > - Select->set_braces(1); > - $$= NULL; > - } > - ; > + // Add the subtree of subquery to the current SELECT_LEX > + SELECT_LEX *curr_sel= Lex->select_stack_head(); > + DBUG_ASSERT(Lex->current_select == curr_sel); > + curr_sel->register_unit($2, &curr_sel->context); > + curr_sel->add_statistics($2); > > -select_derived2: > - { > - LEX *lex= Lex; > - lex->derived_tables|= DERIVED_SUBQUERY; > - if (!lex->expr_allows_subselect || > - lex->sql_command == (int)SQLCOM_PURGE) > - { > - thd->parse_error(); > - MYSQL_YYABORT; > - } > - if (lex->current_select->linkage == GLOBAL_OPTIONS_TYPE || > - mysql_new_select(lex, 1, NULL)) > + Table_ident *ti= new (thd->mem_root) Table_ident($2); > + if (ti == NULL) > MYSQL_YYABORT; > - mysql_init_select(lex); > - lex->current_select->set_linkage(DERIVED_TABLE_TYPE); > - lex->current_select->parsing_place= SELECT_LIST; > - } > - select_options select_item_list > - { > - Select->parsing_place= NO_MATTER; > - } > - opt_table_expression > - ; > - > -get_select_lex: > - /* Empty */ { $$= Select; } > - ; > - > -get_select_lex_derived: > - get_select_lex > - { > - LEX *lex= Lex; > - if ($1->init_nested_join(lex->thd)) > + if (!($$= curr_sel->add_table_to_list(lex->thd, > + ti, $4, 0, > + TL_READ, > MDL_SHARED_READ))) > MYSQL_YYABORT; > } > - ; > - > -select_derived_init: > - { > - LEX *lex= Lex; > - > - TABLE_LIST *embedding= lex->current_select->embedding; > - $$= embedding && > - !embedding->nested_join->join_list.elements; > - /* return true if we are deeply nested */ > - } > ; > > opt_outer: > @@ -11939,9 +12231,14 @@ table_alias: > | '=' > ; > > -opt_table_alias: > +opt_table_alias_clause: > /* empty */ { $$=0; } > - | table_alias ident > + > + | table_alias_clause { $$= $1; } > + ; > + > +table_alias_clause: > + table_alias ident > { > $$= (LEX_CSTRING*) thd->memdup(&$2,sizeof(LEX_STRING)); > if ($$ == NULL) > @@ -12108,7 +12405,7 @@ opt_window_partition_clause: > > opt_window_order_clause: > /* empty */ { } > - | ORDER_SYM BY order_list > + | ORDER_SYM BY order_list { Select->order_list= *($3); } > ; > > opt_window_frame_clause: > @@ -12232,64 +12529,35 @@ alter_order_item: > > opt_order_clause: > /* empty */ > + { $$= NULL; } > | order_clause > + { $$= $1; } > ; > > order_clause: > ORDER_SYM BY > { > - LEX *lex=Lex; > - SELECT_LEX *sel= lex->current_select; > - SELECT_LEX_UNIT *unit= sel-> master_unit(); > - if (sel->linkage != GLOBAL_OPTIONS_TYPE && > - sel->olap != UNSPECIFIED_OLAP_TYPE && > - (sel->linkage != UNION_TYPE || sel->braces)) > - { > - my_error(ER_WRONG_USAGE, MYF(0), > - "CUBE/ROLLUP", "ORDER BY"); > - MYSQL_YYABORT; > - } > - if (lex->sql_command != SQLCOM_ALTER_TABLE && > - !unit->fake_select_lex) > - { > - /* > - A query of the of the form (SELECT ...) ORDER BY > order_list is > - executed in the same way as the query > - SELECT ... ORDER BY order_list > - unless the SELECT construct contains ORDER BY or LIMIT > clauses. > - Otherwise we create a fake SELECT_LEX if it has not > been created > - yet. > - */ > - SELECT_LEX *first_sl= unit->first_select(); > - if (!unit->is_unit_op() && > - (first_sl->order_list.elements || > - first_sl->select_limit) && > - unit->add_fake_select_lex(thd)) > - MYSQL_YYABORT; > - } > - if (sel->master_unit()->is_unit_op() && !sel->braces) > - { > - /* > - At this point we don't know yet whether this is the last > - select in union or not, but we move ORDER BY to > - fake_select_lex anyway. If there would be one more select > - in union mysql_new_select will correctly throw error. > - */ > - DBUG_ASSERT(sel->master_unit()->fake_select_lex); > - lex->current_select= sel->master_unit()->fake_select_lex; > - } > + thd->where= "ORDER clause"; > } > order_list > { > - > + $$= $4; > } > ; > > order_list: > order_list ',' order_ident order_dir > - { if (add_order_to_list(thd, $3,(bool) $4)) MYSQL_YYABORT; } > + { > + $$= $1; > + if (add_to_list(thd, *$$, $3,(bool) $4)) > + MYSQL_YYABORT; > + } > | order_ident order_dir > - { if (add_order_to_list(thd, $1,(bool) $2)) MYSQL_YYABORT; } > + { > + $$= new (thd->mem_root) SQL_I_List<ORDER>(); > + if (add_to_list(thd, *$$, $1, (bool) $2)) > + MYSQL_YYABORT; > + } > ; > > order_dir: > @@ -12299,63 +12567,61 @@ order_dir: > ; > > opt_limit_clause: > - /* empty */ {} > - | limit_clause {} > + /* empty */ > + { $$.empty(); } > + | limit_clause > + { $$= $1; } > ; > > -limit_clause_init: > - LIMIT > - { > - SELECT_LEX *sel= Select; > - if (sel->master_unit()->is_unit_op() && !sel->braces) > - { > - /* Move LIMIT that belongs to UNION to fake_select_lex */ > - Lex->current_select= sel->master_unit()->fake_select_lex; > - DBUG_ASSERT(Select); > - } > - } > - ; > - > limit_clause: > - limit_clause_init limit_options > + LIMIT limit_options > { > - SELECT_LEX *sel= Select; > - if (!sel->select_limit->basic_const_item() || > - sel->select_limit->val_int() > 0) > + $$= $2; > + if (!$$.select_limit->basic_const_item() || > + $$.select_limit->val_int() > 0) > Lex->set_stmt_unsafe(LEX::BINLOG_STMT_UNSAFE_LIMIT); > } > - | limit_clause_init limit_options > + | LIMIT limit_options > ROWS_SYM EXAMINED_SYM limit_rows_option > { > + $$= $2; > Lex->set_stmt_unsafe(LEX::BINLOG_STMT_UNSAFE_LIMIT); > } > - | limit_clause_init ROWS_SYM EXAMINED_SYM limit_rows_option > + | LIMIT ROWS_SYM EXAMINED_SYM limit_rows_option > { > + $$.select_limit= 0; > + $$.offset_limit= 0; > + $$.explicit_limit= 1; > Lex->set_stmt_unsafe(LEX::BINLOG_STMT_UNSAFE_LIMIT); > } > ; > > +opt_global_limit_clause: > + opt_limit_clause > + { > + Select->explicit_limit= $1.explicit_limit; > + Select->select_limit= $1.select_limit; > + Select->offset_limit= $1.offset_limit; > + } > + > limit_options: > limit_option > { > - SELECT_LEX *sel= Select; > - sel->select_limit= $1; > - sel->offset_limit= 0; > - sel->explicit_limit= 1; > + $$.select_limit= $1; > + $$.offset_limit= 0; > + $$.explicit_limit= 1; > } > | limit_option ',' limit_option > { > - SELECT_LEX *sel= Select; > - sel->select_limit= $3; > - sel->offset_limit= $1; > - sel->explicit_limit= 1; > + $$.select_limit= $3; > + $$.offset_limit= $1; > + $$.explicit_limit= 1; > } > | limit_option OFFSET_SYM limit_option > { > - SELECT_LEX *sel= Select; > - sel->select_limit= $1; > - sel->offset_limit= $3; > - sel->explicit_limit= 1; > + $$.select_limit= $1; > + $$.offset_limit= $3; > + $$.explicit_limit= 1; > } > ; > > @@ -12424,6 +12690,66 @@ delete_limit_clause: > | LIMIT limit_option ROWS_SYM EXAMINED_SYM { thd->parse_error(); > MYSQL_YYABORT; } > ; > > +query_expression_tail: > + /* empty */ { $$= NULL; } > + | order_or_limit opt_select_lock_type > + { > + $$= $1; > + $$->lock= $2; > + } > + | order_or_limit procedure_or_into opt_select_lock_type > + { > + $$= $1; > + $$->lock= $3; > + } > + | procedure_or_into opt_select_lock_type > + { > + $$= new(thd->mem_root) Lex_order_limit_lock; > + if (!$$) > + YYABORT; > + $$->order_list= NULL; > + $$->limit.empty(); > + $$->lock= $2; > + } > + | select_lock_type > + { > + $$= new(thd->mem_root) Lex_order_limit_lock; > + if (!$$) > + YYABORT; > + $$->order_list= NULL; > + $$->limit.empty(); > + $$->lock= $1; > + } > + ; > + > +procedure_or_into: > + procedure_clause > + | into > + | procedure_clause into > + ; > + > +order_or_limit: > + order_clause opt_limit_clause > + { > + $$= new(thd->mem_root) Lex_order_limit_lock; > + if (!$$) > + YYABORT; > + $$->order_list= $1; > + $$->limit= $2; > + } > + | limit_clause > + { > + Lex_order_limit_lock *op= $$= new(thd->mem_root) > Lex_order_limit_lock; > + if (!$$) > + YYABORT; > + op->order_list= NULL; > + op->limit= $1; > + $$->order_list= NULL; > + $$->limit= $1; > + } > + ; > + > + > opt_plus: > /* empty */ > | '+' > @@ -12493,14 +12819,11 @@ bool: > | TRUE_SYM { $$= 1; } > | FALSE_SYM { $$= 0; } > > - > procedure_clause: > PROCEDURE_SYM ident /* Procedure name */ > { > LEX *lex=Lex; > > - DBUG_ASSERT(&lex->builtin_select == lex->current_select); > - > lex->proc_list.elements=0; > lex->proc_list.first=0; > lex->proc_list.next= &lex->proc_list.first; > @@ -12520,6 +12843,7 @@ procedure_clause: > parameters are reduced. > */ > Lex->expr_allows_subselect= false; > + Select->options|= OPTION_PROCEDURE_CLAUSE; > } > '(' procedure_list ')' > { > @@ -12600,8 +12924,21 @@ select_outvar: > } > ; > > +opt_into: > + /* empty */ > + | into > + ; > into: > INTO into_destination > + { > + if (!(Select->options & OPTION_INTO_CLAUSE)) > + Select->options|= OPTION_INTO_CLAUSE; > + else > + { > + my_error(ER_CANT_USE_OPTION_HERE, MYF(0), "INTO"); > + MYSQL_YYABORT; > + } > + } > ; > > into_destination: > @@ -12647,10 +12984,15 @@ do: > LEX *lex=Lex; > lex->sql_command = SQLCOM_DO; > mysql_init_select(lex); > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > expr_list > { > Lex->insert_list= $3; > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > } > ; > > @@ -12882,17 +13224,24 @@ insert: > { > LEX *lex= Lex; > lex->sql_command= SQLCOM_INSERT; > - lex->duplicates= DUP_ERROR; > - mysql_init_select(lex); > + lex->duplicates= DUP_ERROR; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + mysql_init_select(lex); > + lex->current_select->parsing_place= BEFORE_OPT_LIST; > } > insert_lock_option > opt_ignore insert2 > { > Select->set_lock_for_tables($3); > - Lex->current_select= &Lex->builtin_select; > + Lex->current_select= Lex->first_select_lex(); > } > insert_field_spec opt_insert_update > - {} > + { > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + } > ; > > replace: > @@ -12901,15 +13250,22 @@ replace: > LEX *lex=Lex; > lex->sql_command = SQLCOM_REPLACE; > lex->duplicates= DUP_REPLACE; > - mysql_init_select(lex); > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + mysql_init_select(lex); > + lex->current_select->parsing_place= BEFORE_OPT_LIST; > } > replace_lock_option insert2 > { > Select->set_lock_for_tables($3); > - Lex->current_select= &Lex->builtin_select; > + Lex->current_select= Lex->first_select_lex(); > } > insert_field_spec > - {} > + { > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + } > ; > > insert_lock_option: > @@ -12955,35 +13311,47 @@ insert_table: > table_name_with_opt_use_partition > { > LEX *lex=Lex; > - lex->field_list.empty(); > + //lex->field_list.empty(); > lex->many_values.empty(); > lex->insert_list=0; > }; > > insert_field_spec: > insert_values {} > - | '(' ')' insert_values {} > - | '(' fields ')' insert_values {} > + | insert_field_list insert_values {} > | SET > { > LEX *lex=Lex; > if (!(lex->insert_list= new (thd->mem_root) List_item) || > lex->many_values.push_back(lex->insert_list, > thd->mem_root)) > MYSQL_YYABORT; > + lex->current_select->parsing_place= NO_MATTER; > } > ident_eq_list > ; > > +insert_field_list: > + LEFT_PAREN_ALT opt_fields ')' > + { > + Lex->current_select->parsing_place= AFTER_LIST; > + } > + ; > + > +opt_fields: > + /* empty */ > + | fields > + ; > + > fields: > fields ',' insert_ident > { Lex->field_list.push_back($3, thd->mem_root); } > | insert_ident { Lex->field_list.push_back($1, thd->mem_root); } > ; > > + > + > insert_values: > - VALUES values_list {} > - | VALUE_SYM values_list {} > - | create_select_query_expression {} > + create_select_query_expression {} > ; > > values_list: > @@ -13093,6 +13461,8 @@ update: > UPDATE_SYM > { > LEX *lex= Lex; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > mysql_init_select(lex); > lex->sql_command= SQLCOM_UPDATE; > lex->duplicates= DUP_ERROR; > @@ -13118,7 +13488,14 @@ update: > */ > Select->set_lock_for_tables($3); > } > - opt_where_clause opt_order_clause delete_limit_clause {} > + opt_where_clause opt_order_clause delete_limit_clause > + { > + if ($10) > + Select->order_list= *($10); > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + } > ; > > update_list: > @@ -13164,6 +13541,8 @@ delete: > mysql_init_select(lex); > YYPS->m_lock_type= TL_WRITE_DEFAULT; > YYPS->m_mdl_type= MDL_SHARED_WRITE; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > > lex->ignore= 0; > lex->builtin_select.init_order(); > @@ -13185,7 +13564,12 @@ single_multi: > } > opt_where_clause opt_order_clause > delete_limit_clause {} > - opt_select_expressions {} > + opt_select_expressions > + { > + if ($6) > + Select->order_list= *($6); > + Lex->pop_select(); //main select > + } > | table_wild_list > { > mysql_init_multi_delete(Lex); > @@ -13196,6 +13580,9 @@ single_multi: > { > if (multi_delete_set_locks_and_link_aux_tables(Lex)) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > } > | FROM table_alias_ref_list > { > @@ -13207,6 +13594,9 @@ single_multi: > { > if (multi_delete_set_locks_and_link_aux_tables(Lex)) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > } > ; > > @@ -13284,6 +13674,7 @@ truncate: > lex->m_sql_cmd= new (thd->mem_root) Sql_cmd_truncate_table(); > if (lex->m_sql_cmd == NULL) > MYSQL_YYABORT; > + Lex->pop_select(); //main select > } > opt_truncate_table_storage_clause { } > ; > @@ -13365,6 +13756,8 @@ show: > LEX *lex=Lex; > lex->wild=0; > lex->ident= null_clex_str; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > mysql_init_select(lex); > lex->current_select->parsing_place= SELECT_LIST; > lex->create_info.init(); > @@ -13372,6 +13765,7 @@ show: > show_param > { > Select->parsing_place= NO_MATTER; > + Lex->pop_select(); //main select > } > ; > > @@ -13387,7 +13781,7 @@ show_param: > { > LEX *lex= Lex; > lex->sql_command= SQLCOM_SHOW_TABLES; > - lex->builtin_select.db= $3; > + lex->first_select_lex()->db= $3; > if (prepare_schema_table(thd, lex, 0, SCH_TABLE_NAMES)) > MYSQL_YYABORT; > } > @@ -13395,7 +13789,7 @@ show_param: > { > LEX *lex= Lex; > lex->sql_command= SQLCOM_SHOW_TRIGGERS; > - lex->builtin_select.db= $3; > + lex->first_select_lex()->db= $3; > if (prepare_schema_table(thd, lex, 0, SCH_TRIGGERS)) > MYSQL_YYABORT; > } > @@ -13403,7 +13797,7 @@ show_param: > { > LEX *lex= Lex; > lex->sql_command= SQLCOM_SHOW_EVENTS; > - lex->builtin_select.db= $2; > + lex->first_select_lex()->db= $2; > if (prepare_schema_table(thd, lex, 0, SCH_EVENTS)) > MYSQL_YYABORT; > } > @@ -13411,7 +13805,7 @@ show_param: > { > LEX *lex= Lex; > lex->sql_command= SQLCOM_SHOW_TABLE_STATUS; > - lex->builtin_select.db= $3; > + lex->first_select_lex()->db= $3; > if (prepare_schema_table(thd, lex, 0, SCH_TABLES)) > MYSQL_YYABORT; > } > @@ -13419,7 +13813,7 @@ show_param: > { > LEX *lex= Lex; > lex->sql_command= SQLCOM_SHOW_OPEN_TABLES; > - lex->builtin_select.db= $3; > + lex->first_select_lex()->db= $3; > if (prepare_schema_table(thd, lex, 0, SCH_OPEN_TABLES)) > MYSQL_YYABORT; > } > @@ -13469,12 +13863,13 @@ show_param: > LEX *lex= Lex; > lex->sql_command= SQLCOM_SHOW_BINLOG_EVENTS; > } > - opt_limit_clause > + opt_global_limit_clause > | RELAYLOG_SYM optional_connection_name EVENTS_SYM binlog_in > binlog_from > { > LEX *lex= Lex; > lex->sql_command= SQLCOM_SHOW_RELAYLOG_EVENTS; > - } opt_limit_clause > + } > + opt_global_limit_clause > | keys_or_index from_or_in table_ident opt_db opt_where_clause > { > LEX *lex= Lex; > @@ -13516,13 +13911,13 @@ show_param: > LEX_CSTRING var= {STRING_WITH_LEN("error_count")}; > (void) create_select_for_variable(thd, &var); > } > - | WARNINGS opt_limit_clause > + | WARNINGS opt_global_limit_clause > { Lex->sql_command = SQLCOM_SHOW_WARNS;} > - | ERRORS opt_limit_clause > + | ERRORS opt_global_limit_clause > { Lex->sql_command = SQLCOM_SHOW_ERRORS;} > | PROFILES_SYM > { Lex->sql_command = SQLCOM_SHOW_PROFILES; } > - | PROFILE_SYM opt_profile_defs opt_profile_args opt_limit_clause > + | PROFILE_SYM opt_profile_defs opt_profile_args > opt_global_limit_clause > { > LEX *lex= Lex; > lex->sql_command= SQLCOM_SHOW_PROFILE; > @@ -13829,7 +14224,7 @@ describe: > explainable_command > { > LEX *lex=Lex; > - lex->builtin_select.options|= SELECT_DESCRIBE; > + lex->first_select_lex()->options|= SELECT_DESCRIBE; > } > ; > > @@ -13855,6 +14250,8 @@ analyze_stmt_command: > > opt_extended_describe: > EXTENDED_SYM { Lex->describe|= DESCRIBE_EXTENDED; } > + | EXTENDED_SYM ALL > + { Lex->describe|= DESCRIBE_EXTENDED | DESCRIBE_EXTENDED2; } > | PARTITIONS_SYM { Lex->describe|= DESCRIBE_PARTITIONS; } > | opt_format_json {} > ; > @@ -13897,8 +14294,7 @@ flush: > lex->type= 0; > lex->no_write_to_binlog= $2; > } > - flush_options > - {} > + flush_options {} > ; > > flush_options: > @@ -13915,6 +14311,7 @@ flush_options: > opt_table_list opt_flush_lock > {} > | flush_options_list > + {} > ; > > opt_flush_lock: > @@ -14070,9 +14467,13 @@ purge: > LEX *lex=Lex; > lex->type=0; > lex->sql_command = SQLCOM_PURGE; > + if (lex->main_select_push()) > + MYSQL_YYABORT; > } > purge_options > - {} > + { > + Lex->pop_select(); //main select > + } > ; > > purge_options: > @@ -14090,6 +14491,8 @@ purge_option: > lex->value_list.empty(); > lex->value_list.push_front($2, thd->mem_root); > lex->sql_command= SQLCOM_PURGE_BEFORE; > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > } > ; > > @@ -14099,6 +14502,8 @@ kill: > KILL_SYM > { > LEX *lex=Lex; > + if (lex->main_select_push()) > + YYABORT; > lex->value_list.empty(); > lex->users_list.empty(); > lex->sql_command= SQLCOM_KILL; > @@ -14107,6 +14512,7 @@ kill: > kill_type kill_option kill_expr > { > Lex->kill_signal= (killed_state) ($3 | $4); > + Lex->pop_select(); //main select > } > ; > > @@ -14167,6 +14573,8 @@ load: > $2 == FILETYPE_CSV ? "LOAD DATA" : "LOAD XML"); > MYSQL_YYABORT; > } > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > load_data_lock opt_local INFILE TEXT_STRING_filesystem > { > @@ -14193,7 +14601,11 @@ load: > opt_xml_rows_identified_by > opt_field_term opt_line_term opt_ignore_lines > opt_field_or_var_spec > opt_load_data_set_spec > - {} > + { > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + } > ; > > data_or_xml: > @@ -14597,17 +15009,21 @@ opt_with_clause: > > > with_clause: > - WITH opt_recursive > + WITH opt_recursive > { > + LEX *lex= Lex; > With_clause *with_clause= > new With_clause($2, Lex->curr_with_clause); > if (with_clause == NULL) > MYSQL_YYABORT; > - Lex->derived_tables|= DERIVED_WITH; > - Lex->curr_with_clause= with_clause; > + lex->derived_tables|= DERIVED_WITH; > + lex->curr_with_clause= with_clause; > with_clause->add_to_list(Lex->with_clauses_list_last_next); > + if (lex->current_select && > + lex->current_select->parsing_place == BEFORE_OPT_LIST) > + lex->current_select->parsing_place= NO_MATTER; > } > - with_list > + with_list > { > $$= Lex->curr_with_clause; > Lex->curr_with_clause= Lex->curr_with_clause->pop(); > @@ -14636,9 +15052,9 @@ with_list_element: > MYSQL_YYABORT; > Lex->with_column_list.empty(); > } > - AS '(' remember_name subselect remember_end ')' > + AS '(' remember_name query_expression remember_end ')' > { > - With_element *elem= new With_element($1, *$2, > $7->master_unit()); > + With_element *elem= new With_element($1, *$2, $7); > if (elem == NULL || Lex->curr_with_clause->add_with_element(elem)) > MYSQL_YYABORT; > if (elem->set_unparsed_spec(thd, $6+1, $8)) > @@ -15589,14 +16005,22 @@ set: > SET > { > LEX *lex=Lex; > + if (lex->main_select_push()) > + MYSQL_YYABORT; > lex->set_stmt_init(); > lex->var_list.empty(); > sp_create_assignment_lex(thd, yychar == YYEMPTY); > } > start_option_value_list > - {} > + { > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + } > | SET STATEMENT_SYM > { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > Lex->set_stmt_init(); > } > set_stmt_option_value_following_option_type_list > @@ -15606,6 +16030,9 @@ set: > my_yyabort_error((ER_SUBQUERIES_NOT_SUPPORTED, MYF(0), > "SET STATEMENT")); > lex->stmt_var_list= lex->var_list; > lex->var_list.empty(); > + Lex->pop_select(); //main select > + if (Lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > } > FOR_SYM verb_clause > {} > @@ -16038,9 +16465,13 @@ lock: > if (lex->sphead) > my_yyabort_error((ER_SP_BADSTATEMENT, MYF(0), "LOCK")); > lex->sql_command= SQLCOM_LOCK_TABLES; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > table_lock_list opt_lock_wait_timeout > - {} > + { > + Lex->pop_select(); //main select > + } > ; > > opt_lock_wait_timeout: > @@ -16071,7 +16502,7 @@ table_lock_list: > ; > > table_lock: > - table_ident opt_table_alias lock_option > + table_ident opt_table_alias_clause lock_option > { > thr_lock_type lock_type= (thr_lock_type) $3; > bool lock_for_write= (lock_type >= TL_WRITE_ALLOW_WRITE); > @@ -16105,9 +16536,13 @@ unlock: > if (lex->sphead) > my_yyabort_error((ER_SP_BADSTATEMENT, MYF(0), "UNLOCK")); > lex->sql_command= SQLCOM_UNLOCK_TABLES; > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > } > table_or_tables > - {} > + { > + Lex->pop_select(); //main select > + } > ; > > /* > @@ -16115,25 +16550,36 @@ unlock: > */ > > handler: > - HANDLER_SYM table_ident OPEN_SYM opt_table_alias > + HANDLER_SYM > + { > + if (Lex->main_select_push()) > + MYSQL_YYABORT; > + } > + handler_tail > + { > + Lex->pop_select(); //main select > + } > + > +handler_tail: > + table_ident OPEN_SYM opt_table_alias_clause > { > LEX *lex= Lex; > if (lex->sphead) > my_yyabort_error((ER_SP_BADSTATEMENT, MYF(0), "HANDLER")); > lex->sql_command = SQLCOM_HA_OPEN; > - if (!lex->current_select->add_table_to_list(thd, $2, $4, 0)) > + if (!lex->current_select->add_table_to_list(thd, $1, $3, 0)) > MYSQL_YYABORT; > } > - | HANDLER_SYM table_ident_nodb CLOSE_SYM > + | table_ident_nodb CLOSE_SYM > { > LEX *lex= Lex; > if (lex->sphead) > my_yyabort_error((ER_SP_BADSTATEMENT, MYF(0), "HANDLER")); > lex->sql_command = SQLCOM_HA_CLOSE; > - if (!lex->current_select->add_table_to_list(thd, $2, 0, 0)) > + if (!lex->current_select->add_table_to_list(thd, $1, 0, 0)) > MYSQL_YYABORT; > } > - | HANDLER_SYM table_ident_nodb READ_SYM > + | table_ident_nodb READ_SYM > { > LEX *lex=Lex; > if (lex->sphead) > @@ -16141,20 +16587,24 @@ handler: > lex->expr_allows_subselect= FALSE; > lex->sql_command = SQLCOM_HA_READ; > lex->ha_rkey_mode= HA_READ_KEY_EXACT; /* Avoid purify > warnings */ > - Item *one= new (thd->mem_root) Item_int(thd, (int32) 1); > - if (one == NULL) > - MYSQL_YYABORT; > - lex->current_select->select_limit= one; > - lex->current_select->offset_limit= 0; > - lex->limit_rows_examined= 0; > - if (!lex->current_select->add_table_to_list(thd, $2, 0, 0)) > + if (!lex->current_select->add_table_to_list(thd, $1, 0, 0)) > MYSQL_YYABORT; > } > - handler_read_or_scan opt_where_clause opt_limit_clause > + handler_read_or_scan opt_where_clause opt_global_limit_clause > { > - Lex->expr_allows_subselect= TRUE; > + LEX *lex=Lex; > + lex->expr_allows_subselect= TRUE; > + if (!lex->current_select->explicit_limit) > + { > + Item *one= new (thd->mem_root) Item_int(thd, (int32) 1); > + if (one == NULL) > + MYSQL_YYABORT; > + lex->current_select->select_limit= one; > + lex->current_select->offset_limit= 0; > + lex->limit_rows_examined= 0; > + } > /* Stored functions are not supported for HANDLER READ. */ > - if (Lex->uses_stored_routines()) > + if (lex->uses_stored_routines()) > { > my_error(ER_NOT_SUPPORTED_YET, MYF(0), > "stored functions in HANDLER ... READ"); > @@ -16800,83 +17250,16 @@ release: > */ > > unit_type_decl: > - UNION_SYM > - { $$= UNION_TYPE; } > + UNION_SYM union_option > + { $$.unit_type= UNION_TYPE; $$.distinct= $2; } > | INTERSECT_SYM > - { $$= INTERSECT_TYPE; } > + { $$.unit_type= INTERSECT_TYPE; $$.distinct= 1; } > | EXCEPT_SYM > - { $$= EXCEPT_TYPE; } > - > - > -union_clause: > - /* empty */ {} > - | union_list > - ; > - > -union_list: > - unit_type_decl union_option > - { > - if (Lex->add_select_to_union_list((bool)$2, $1, TRUE)) > - MYSQL_YYABORT; > - } > - union_list_part2 > - { > - /* > - Remove from the name resolution context stack the context > of the > - last select in the union. > - */ > - Lex->pop_context(); > - } > - ; > - > -union_list_view: > - unit_type_decl union_option > - { > - if (Lex->add_select_to_union_list((bool)$2, $1, TRUE)) > - MYSQL_YYABORT; > - } > - query_expression_body_view > - { > - Lex->pop_context(); > - } > - ; > - > -union_order_or_limit: > - { > - LEX *lex= thd->lex; > - DBUG_ASSERT(lex->current_select->linkage != > GLOBAL_OPTIONS_TYPE); > - SELECT_LEX *sel= lex->current_select; > - SELECT_LEX_UNIT *unit= sel->master_unit(); > - SELECT_LEX *fake= unit->fake_select_lex; > - if (fake) > - { > - fake->no_table_names_allowed= 1; > - lex->current_select= fake; > - } > - thd->where= "global ORDER clause"; > - } > - order_or_limit > - { > - thd->lex->current_select->no_table_names_allowed= 0; > - thd->where= ""; > - } > - ; > - > -order_or_limit: > - order_clause opt_limit_clause > - | limit_clause > - ; > + { $$.unit_type= EXCEPT_TYPE; $$.distinct= 1; } > > /* > Start a UNION, for non-top level query expressions. > */ > -union_head_non_top: > - unit_type_decl union_option > - { > - if (Lex->add_select_to_union_list((bool)$2, $1, FALSE)) > - MYSQL_YYABORT; > - } > - ; > > union_option: > /* empty */ { $$=1; } > @@ -16884,110 +17267,10 @@ union_option: > | ALL { $$=0; } > ; > > -/* > - Corresponds to the SQL Standard > - <query specification> ::= > - SELECT [ <set quantifier> ] <select list> <table expression> > - > - Notes: > - - We allow more options in addition to <set quantifier> > - - <table expression> is optional in MariaDB > -*/ > -query_specification: > - SELECT_SYM select_init2_derived opt_table_expression > - { > - $$= Lex->current_select->master_unit()->first_select(); > - } > - ; > - > -query_term_union_not_ready: > - query_specification order_or_limit opt_select_lock_type { $$= > $1; } > - | '(' select_paren_derived ')' union_order_or_limit { $$= > $2; } > - ; > - > -query_term_union_ready: > - query_specification opt_select_lock_type { $$= > $1; } > - | '(' select_paren_derived ')' { $$= > $2; } > - ; > - > -query_expression_body: > - query_term_union_not_ready { > $$= $1; } > - | query_term_union_ready { > $$= $1; } > - | query_term_union_ready union_list_derived { > $$= $1; } > - ; > - > -/* Corresponds to <query expression> in the SQL:2003 standard. */ > -subselect: > - subselect_start opt_with_clause query_expression_body > subselect_end > - { > - $3->set_with_clause($2); > - $$= $3; > - } > - ; > - > -subselect_start: > - { > - LEX *lex=Lex; > - if (!lex->expr_allows_subselect || > - lex->sql_command == (int)SQLCOM_PURGE) > - { > - thd->parse_error(); > - MYSQL_YYABORT; > - } > - /* > - we are making a "derived table" for the parenthesis > - as we need to have a lex level to fit the union > - after the parenthesis, e.g. > - (SELECT .. ) UNION ... becomes > - SELECT * FROM ((SELECT ...) UNION ...) > - */ > - if (mysql_new_select(Lex, 1, NULL)) > - MYSQL_YYABORT; > - } > - ; > - > -subselect_end: > - { > - LEX *lex=Lex; > - > - lex->check_automatic_up(UNSPECIFIED_TYPE); > - lex->pop_context(); > - SELECT_LEX *child= lex->current_select; > - lex->current_select = > lex->current_select->return_after_parsing(); > - lex->nest_level--; > - lex->current_select->n_child_sum_items += child->n_sum_items; > - /* > - A subselect can add fields to an outer select. Reserve > space for > - them. > - */ > - lex->current_select->select_n_where_fields+= > - child->select_n_where_fields; > - > - /* > - Aggregate functions in having clause may add fields to an > outer > - select. Count them also. > - */ > - lex->current_select->select_n_having_items+= > - child->select_n_having_items; > - } > - ; > - > -opt_query_expression_options: > - /* empty */ > - | query_expression_option_list > - ; > - > -query_expression_option_list: > - query_expression_option_list query_expression_option > - | query_expression_option > - ; > - > query_expression_option: > STRAIGHT_JOIN { Select->options|= SELECT_STRAIGHT_JOIN; } > | HIGH_PRIORITY > { > - if (check_simple_select()) > - MYSQL_YYABORT; > YYPS->m_lock_type= TL_READ_HIGH_PRIORITY; > YYPS->m_mdl_type= MDL_SHARED_READ; > Select->options|= SELECT_HIGH_PRIORITY; > @@ -16996,18 +17279,8 @@ query_expression_option: > | UNIQUE_SYM { Select->options|= SELECT_DISTINCT; } > | SQL_SMALL_RESULT { Select->options|= SELECT_SMALL_RESULT; } > | SQL_BIG_RESULT { Select->options|= SELECT_BIG_RESULT; } > - | SQL_BUFFER_RESULT > - { > - if (check_simple_select()) > - MYSQL_YYABORT; > - Select->options|= OPTION_BUFFER_RESULT; > - } > - | SQL_CALC_FOUND_ROWS > - { > - if (check_simple_select()) > - MYSQL_YYABORT; > - Select->options|= OPTION_FOUND_ROWS; > - } > + | SQL_BUFFER_RESULT { Select->options|= OPTION_BUFFER_RESULT; } > + | SQL_CALC_FOUND_ROWS { Select->options|= OPTION_FOUND_ROWS; } > | ALL { Select->options|= SELECT_ALL; } > ; > > @@ -17095,32 +17368,28 @@ view_select: > lex->parsing_options.allows_variable= FALSE; > lex->create_view->select.str= (char *) YYLIP->get_cpp_ptr(); > } > - opt_with_clause query_expression_body_view view_check_option > + query_expression > + view_check_option > { > LEX *lex= Lex; > + SQL_I_List<TABLE_LIST> *save= > &lex->first_select_lex()->table_list; > + lex->set_main_unit($2); > + if (lex->check_main_unit_semantics()) > + MYSQL_YYABORT; > + lex->first_select_lex()->table_list.push_front(save); > + lex->current_select= Lex->first_select_lex(); > size_t len= YYLIP->get_cpp_ptr() - > lex->create_view->select.str; > void *create_view_select= > thd->memdup(lex->create_view->select.str, len); > lex->create_view->select.length= len; > lex->create_view->select.str= (char *) create_view_select; > + size_t not_used; > trim_whitespace(thd->charset(), > - &lex->create_view->select); > - lex->create_view->check= $4; > + &lex->create_view->select, &not_used); > + lex->create_view->check= $3; > lex->parsing_options.allows_variable= TRUE; > - lex->current_select->set_with_clause($2); > } > ; > > -/* > - SQL Standard <query expression body> for VIEWs. > - Does not include INTO and PROCEDURE clauses. > -*/ > -query_expression_body_view: > - SELECT_SYM select_options_and_item_list select_init3_view > - | '(' select_paren_view ')' > - | '(' select_paren_view ')' union_order_or_limit > - | '(' select_paren_view ')' union_list_view > - ; > - > view_check_option: > /* empty */ { $$= VIEW_CHECK_NONE; } > | WITH CHECK_SYM OPTION { $$= VIEW_CHECK_CASCADED; } > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits >

2 5

[Maria-developers] MDEV-14448
by Badrul Chowdhury 16 Apr '18

16 Apr '18

Hello, Is anyone working on https://jira.mariadb.org/browse/MDEV-14448? Pointers to documentation on the procedure to adopt upstream fixes would be appreciated also. Thanks, Badrul

2 2

Re: [Maria-developers] Observation for CHECK_CONSTRAINTS MDEV 14474
by Vicențiu Ciorbaru 14 Apr '18

14 Apr '18

Hi Anel! I've reviewed your patch and generally things look good. You do some good analysis usually and I think we should highlight your work to the broader developer scene. I've thus CC'ed this email to the maria-developers mailing list. This gives us a chance to get feedback from outside, particularly as this feature is of general interest to the community. Please include the .result files in future versions as it makes reviewing a lot easier. Serg: coding style and implementation aside, I'll do the review on that bit, please have a look at Anel's patch and suggest if you think the current table header for I_S.check_constraints is good enough for our needs or if you have more input on this task. Relevant bit: +ST_FIELD_INFO check_constraints_fields_info[]= +{ + {"CONSTRAINT_CATALOG", FN_REFLEN, MYSQL_TYPE_STRING, 0, 0, 0, OPEN_FULL_TABLE}, + {"CONSTRAINT_SCHEMA", NAME_CHAR_LEN, MYSQL_TYPE_STRING, 0, 0, 0, + OPEN_FULL_TABLE}, + {"CONSTRAINT_NAME", NAME_CHAR_LEN, MYSQL_TYPE_STRING, 0, 0, 0, + OPEN_FULL_TABLE}, + {"TABLE_SCHEMA", NAME_CHAR_LEN, MYSQL_TYPE_STRING, 0, 0, 0, OPEN_FULL_TABLE}, + {"TABLE_NAME", NAME_CHAR_LEN, MYSQL_TYPE_STRING, 0, 0, 0, OPEN_FULL_TABLE}, + {"CHECK_CLAUSE", NAME_CHAR_LEN, MYSQL_TYPE_STRING, 0, 0, 0, + OPEN_FULL_TABLE}, + {0, 0, MYSQL_TYPE_STRING, 0, 0, 0, SKIP_OPEN_TABLE} +}; Vicențiu On Thu, 29 Mar 2018 at 12:01 Anel Husakovic <anel(a)mariadb.org> wrote: > Hi Vin, > > I'm pleased to announce my new patch prototype for MDEV 14474 where are > implemented field and table constraints: > > https://github.com/an3l/server/commit/f92fd77d2695be52faf280245faa014c44617… > > According to the standard there are only 4 column to represent. > CONSTRAINT_CATALOG, CONSTRAINT_SCHEMA, CONSTRAINT_NAME,CHECK_CLAUSE > But because Serg mentioned that alternative would be to change > *information_schema.table_constraints* with adding only 1 column ( > *check_clause*) I also used 2 additional columns which are represented > also in *table_constraints *and they are TABLE_SHEMA , TABLE_NAME. > > Function of interest is: get_check_constraints_record > > Also I added 2 test cases: > > One that works with only 1 table: MDEV14474-one_table_play.test > <https://github.com/an3l/server/commit/f92fd77d2695be52faf280245faa014c44617…> > Second that works with more tables: MDEV14474-I_S_check_constraint.test > <https://github.com/an3l/server/commit/f92fd77d2695be52faf280245faa014c44617…> > > Last night I had a problem with field constraints. > As a reference I used only way how currently all check constraints are > shown and its obtained via* show create table t;* > > Bellow is shown that field constraints were not visible as well as that > name used for them is *NULL*, at least this is how *show create table* > represented them ( there is no way to give a name to a field constraint* > for example: f int constraint name check(f<0)*). According to my opinion, > because this constraint belongs to the field, it should be also called as a > name of a field and not NULL. > As an argument to my opinion here is mariadb kb > https://mariadb.com/kb/en/library/constraint/ > Please take a look at *constraint **name*, which is field constraint and > error message. > > INSERT INTO t2(name, start_date, end_date) VALUES('Io', '2003-12-15', '2014-11-09');ERROR 4022 (23000): CONSTRAINT `name` failed for `test`.`t2` > > If the name of constraint = name of the field , than definitely it is not > consistent with* show create table* output, and *show create table* > output *needs to be changed*, , but this is question that I would like > to discuss with you ?! > > When you try to use show create table x > [image: image.png] > > > I'm waiting on your reply for global temporary tables as well as > check_constraints and how to proceed further > > *Not understand yet a concept*: > > In *sql/table.h* > There is a struct* st_schema_table *which has fill data and process data. > > How they work, and how to use that on other database for example PSI. > > Regards, > > Anel >

2 1

Re: [Maria-developers] [Commits] a74e3ef17e7: MDEV-14551 Can't find record in table on multi-table update with ORDER BY
by Sergey Petrunia 13 Apr '18

13 Apr '18

Hi Serg, Please find some cosmetic input below. (additionally, we've found a failing example while chatting) On Tue, Apr 10, 2018 at 02:10:17PM +0200, serg(a)mariadb.org wrote: > revision-id: a74e3ef17e7a8693d68001290a8d91b5b206804d (mariadb-10.3.5-138-ga74e3ef17e7) > parent(s): 9bd3af97dffa411657879bbdfd4dfef38c99f7cc > author: Sergei Golubchik > committer: Sergei Golubchik > timestamp: 2018-04-10 14:09:17 +0200 > message: > > MDEV-14551 Can't find record in table on multi-table update with ORDER BY ... > diff --git a/mysql-test/main/multi_update.test b/mysql-test/main/multi_update.test > index 5feebe87a5a..96e78dab82b 100644 > --- a/mysql-test/main/multi_update.test > +++ b/mysql-test/main/multi_update.test > @@ -914,3 +914,14 @@ update t1 set c1=NULL; > update t1, t2 set t1.c1=t2.c3 where t1.c3=t2.c3 order by t1.c3 desc limit 2; > select * from t1; > drop table t1, t2; > + > +# > +# MDEV-14551 Can't find record in table on multi-table update with ORDER BY > +# > +CREATE TABLE t1 (i INT) ENGINE=MEMORY; > +INSERT t1 VALUES (1),(2); > +CREATE TABLE t2 (f INT) ENGINE=MyISAM; > +INSERT t2 VALUES (1),(2); > +UPDATE t1, t2 SET f = 126 ORDER BY f LIMIT 2; > +SELECT * FROM t2; > +DROP TABLE t1, t2; > diff --git a/sql/item.h b/sql/item.h > index 9574bdc63bf..e391e7810c4 100644 > --- a/sql/item.h > +++ b/sql/item.h > @@ -2023,6 +2023,7 @@ class Item: public Value_source, > { > marker &= ~EXTRACTION_MASK; > } > + virtual TABLE *rowid_table() const { return 0; } I am concerned about this approach bloating class Item's vtable. There is (and likely will ever be) only one class with a different implementation, the check for this is made only in one place - why not check Item's type() or functype(), etc? > }; > > MEM_ROOT *get_thd_memroot(THD *thd); > diff --git a/sql/item_strfunc.cc b/sql/item_strfunc.cc > index c6b0ae7fba8..9097ffca867 100644 > --- a/sql/item_strfunc.cc > +++ b/sql/item_strfunc.cc > @@ -5204,3 +5204,23 @@ String *Item_func_dyncol_list::val_str(String *str) > my_free(names); > return NULL; > } > + > +Item_temptable_rowid::Item_temptable_rowid(TABLE *table_arg) > + : Item_str_func(table_arg->in_use), table(table_arg) > +{ > + max_length= table->file->ref_length; > +} > + > +void Item_temptable_rowid::fix_length_and_dec() > +{ > + used_tables_cache= table->map; > + const_item_cache= false; > +} > + > +String *Item_temptable_rowid::val_str(String *str) > +{ > + if (!((null_value= table->null_row))) > + table->file->position(table->record[0]); > + str_value.set((char*)(table->file->ref), max_length, &my_charset_bin); > + return &str_value; > +} > diff --git a/sql/item_strfunc.h b/sql/item_strfunc.h > index 18cda491efd..49de5568696 100644 > --- a/sql/item_strfunc.h > +++ b/sql/item_strfunc.h > @@ -1748,5 +1748,20 @@ class Item_func_dyncol_list: public Item_str_func > { return get_item_copy<Item_func_dyncol_list>(thd, this); } > }; > > -#endif /* ITEM_STRFUNC_INCLUDED */ Please add a note that this is not the "_rowid" that we support in the parser. > +class Item_temptable_rowid :public Item_str_func > +{ > + TABLE *table; > +public: > + Item_temptable_rowid(TABLE *table_arg); > + const Type_handler *type_handler() const { return &type_handler_string; } > + Field *create_tmp_field(bool group, TABLE *table) > + { return create_table_field_from_handler(table); } > + String *val_str(String *str); > + const char *func_name() const { return "<rowid>"; } > + void fix_length_and_dec(); > + Item *get_copy(THD *thd) > + { return get_item_copy<Item_temptable_rowid>(thd, this); } > + TABLE *rowid_table() const { return table; } > +}; > > +#endif /* ITEM_STRFUNC_INCLUDED */ ... > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index 796ea569e64..22301319680 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -2650,6 +2650,21 @@ bool JOIN::add_having_as_table_cond(JOIN_TAB *tab) > } > > > +bool JOIN::add_fields_for_current_rowid(JOIN_TAB *cur, List<Item> *table_fields) I think ths is actually "current rowids", in plural. As the loop shows, there are multiple current rowids (from different tables). > +{ > + for (JOIN_TAB *tab=join_tab; tab < cur; tab++) Please add a comment: this will not walk into semi-join materialization nests but this is ok because we will never need to save current rowids for those. > + { > + if (!tab->keep_current_rowid) > + continue; > + Item *item= new (thd->mem_root) Item_temptable_rowid(tab->table); > + item->fix_fields(thd, 0); > + table_fields->push_back(item, thd->mem_root); > + cur->tmp_table_param->func_count++; > + } > + return 0; > +} > + > + > /** > Set info for aggregation tables > > diff --git a/sql/sql_select.h b/sql/sql_select.h > index f8911fbba01..1da87bb9d50 100644 > --- a/sql/sql_select.h > +++ b/sql/sql_select.h > @@ -1438,6 +1438,9 @@ class JOIN :public Sql_alloc > > enum { QEP_NOT_PRESENT_YET, QEP_AVAILABLE, QEP_DELETED} have_query_plan; > > + // if keep_current_rowid=true, whether they should be saved in temporary table > + bool tmp_table_keep_current_rowid; Same input as with add_fields_for_current_rowid - use plural. > + > /* > Additional WHERE and HAVING predicates to be considered for IN=>EXISTS > subquery transformation of a JOIN object. > diff --git a/sql/sql_union.cc b/sql/sql_union.cc > index 0149c2848c2..0e29f817e56 100644 > --- a/sql/sql_union.cc > +++ b/sql/sql_union.cc > @@ -507,14 +507,14 @@ void select_union_recursive::cleanup() > bool select_union_direct::change_result(select_result *new_result) > { > result= new_result; > - return (result->prepare(unit->types, unit) || result->prepare2()); > + return (result->prepare(unit->types, unit) || result->prepare2(0)); Do we really continue to use '0' when 'NULL' should be used? I prefer "NULL". > } > > > bool select_union_direct::postponed_prepare(List<Item> &types) > { > if (result != NULL) > - return (result->prepare(types, unit) || result->prepare2()); > + return (result->prepare(types, unit) || result->prepare2(0)); > else > return false; > } > diff --git a/sql/sql_update.cc b/sql/sql_update.cc > index 38638d3aa1d..4ae7e33b4e3 100644 > --- a/sql/sql_update.cc > +++ b/sql/sql_update.cc > @@ -2205,10 +2195,41 @@ multi_update::initialize_tables(JOIN *join) > DBUG_RETURN(1); > tmp_tables[cnt]->file->extra(HA_EXTRA_WRITE_CACHE); > } > + join->tmp_table_keep_current_rowid= TRUE; > DBUG_RETURN(0); > } > (Follwing explanations on Slack) Please add a comment there, saying that Here we walk through the aggegation temptable to find where rowids are stored. And then for each of the multi-update tables, we get them to copy the data from the rowid field in the aggregation temptable. > + > +int multi_update::prepare2(JOIN *join) > +{ > + if (!join->need_tmp || !join->tmp_table_keep_current_rowid) > + return 0; > + > + // there cannot be many tmp tables in multi-update > + JOIN_TAB *tmptab= join->join_tab + join->exec_join_tab_cnt(); > + > + for (Item **it= tmptab->tmp_table_param->items_to_copy; *it ; it++) > + { > + TABLE *tbl= (*it)->rowid_table(); > + if (!tbl) > + continue; > + for (uint i= 0; i < table_count; i++) > + { > + for (Item **it2= tmp_table_param[i].items_to_copy; *it2; it2++) > + { > + if ((*it2)->rowid_table() != tbl) > + continue; > + *it2= new (thd->mem_root) Item_field(thd, (*it)->get_tmp_table_field()); > + if (!*it2) > + return 1; > + } > + } > + } > + return 0; > +} > + > + > multi_update::~multi_update() > { > TABLE_LIST *table; BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

2 5

[Maria-developers] FOR UPDATE behavior change in 10.3-MDEV-11953
by Alexander Barkov 12 Apr '18

12 Apr '18

Hi Sanja, and Igor, (resending with Igor on CC) I noticed a FOR UPDATE behavior change in 10.3-MDEV-11953 In 10.3 I run this script in a client session: CREATE OR REPLACE TABLE t1 (a INT NOT NULL PRIMARY KEY); CREATE OR REPLACE TABLE t2 (a INT NOT NULL PRIMARY KEY); INSERT INTO t1 VALUES (1); INSERT INTO t2 VALUES (2); BEGIN; SELECT * FROM t1 UNION SELECT * FROM t2 FOR UPDATE; Notice no COMMIT yet! It returns this result: +---+ | a | +---+ | 1 | | 2 | +---+ Now I open a new console, start a new client session and run this script: BEGIN; SELECT * FROM t2 FOR UPDATE; The second session gets locked, as expected. Now I return to the first console and run "COMMIT;". The second console gets unlocked and returns +---+ | a | +---+ | 2 | +---+ Now if I do the same in 10.3-MDEV-11953, the second session does not get locked, it returns the result immediately, even before I typed "COMMIT;" in the first session. Note, if in 10.3-MDEV-11953 I start a transaction with a simpler query (without UNION) in the first session: BEGIN; SELECT * FROM t2 FOR UPDATE; and run the same script in the second session BEGIN; SELECT * FROM t2 FOR UPDATE; then the second session gets locked as expected. Is this behavior change expected? Thanks!

2 1

Re: [Maria-developers] FOR UPDATE behavior change in 10.3-MDEV-11953
by Alexander Barkov 12 Apr '18

12 Apr '18

Hi Igor, Please do "reply all" when replying. On 04/12/2018 01:28 AM, Igor Babaev wrote: > Alexander, > > If you have the same behaviour you have to use > > SELECT * FROM t1 UNION (SELECT * FROM t2 FOR UPDATE). Thanks. This worked fine for me. > > Otherwise it's considered as > > (SELECT * FROM t1 UNION SELECT * FROM t2 FOR UPDATE). What does this syntax mean 10.3-MDEV-11953? Which records should be locked? From t1, from t2, or from both? It seems that it locks nothing. If so, shouldn't this be disallowed? This query (without parentheses) also locks nothing: SELECT * FROM t1 UNION SELECT * FROM t2 FOR UPDATE; > > MySQL manual says that 'FOR UPDATE' must follow ORDER BY, > > but it does not tell you how to interpret FOR UPDATE in unions. The old behavior was to interpret UNION parts separately. This locked only t2 in 10.3: SELECT * FROM t1 UNION SELECT * FROM t2 FOR UPDATE; In 10.3-MDEV-11953 it locks nothing. Upgrade looks dangerous. Can we do this way: 1. In 10.4: disallow FOR UPDATE applied to the entire UNION. 2. In 10.3: Add a warning which tells that this feature is deprecated and suggests to use parentheses around union parts when a non-parenthesized union term is used with FOR UPDATE ? > > Anyway you can't use anymore: > SELECT c FROM t1 ORDER BY a > UNION > SELECT c FROM t2 ORDER BY b > ORDER BY c. > > You have to put it so: > (SELECT c FROM t1 ORDER BY a) > UNION > (SELECT c FROM t2 ORDER BY b) > ORDER BY c. > > (see the standard). I like this change. > > It means that the ORDER BY clause followed a union relates to the union. > The same is true for 'FOR UPDATE' because it follows optional ORDER BY/ > LIMIT clauses. > > > Regards, > Igor. > > > On 04/11/2018 09:10 AM, Alexander Barkov wrote: >> Hi Sanja, and Igor, >> >> >> (resending with Igor on CC) >> >> >> I noticed a FOR UPDATE behavior change in 10.3-MDEV-11953 >> >> In 10.3 I run this script in a client session: >> >> CREATE OR REPLACE TABLE t1 (a INT NOT NULL PRIMARY KEY); >> CREATE OR REPLACE TABLE t2 (a INT NOT NULL PRIMARY KEY); >> INSERT INTO t1 VALUES (1); >> INSERT INTO t2 VALUES (2); >> BEGIN; >> SELECT * FROM t1 UNION SELECT * FROM t2 FOR UPDATE; >> >> Notice no COMMIT yet! It returns this result: >> +---+ >> | a | >> +---+ >> | 1 | >> | 2 | >> +---+ >> >> Now I open a new console, start a new client session and run this script: >> >> BEGIN; >> SELECT * FROM t2 FOR UPDATE; >> >> The second session gets locked, as expected. >> >> Now I return to the first console and run "COMMIT;". >> The second console gets unlocked and returns >> +---+ >> | a | >> +---+ >> | 2 | >> +---+ >> >> >> Now if I do the same in 10.3-MDEV-11953, the second session does not get >> locked, it returns the result immediately, even before I typed "COMMIT;" >> in the first session. >> >> >> Note, if in 10.3-MDEV-11953 I start a transaction with a simpler query >> (without UNION) in the first session: >> >> BEGIN; >> SELECT * FROM t2 FOR UPDATE; >> >> and run the same script in the second session >> >> BEGIN; >> SELECT * FROM t2 FOR UPDATE; >> >> then the second session gets locked as expected. >> >> >> Is this behavior change expected? >> >> >> Thanks! >> >> _______________________________________________ >> Mailing list: https://launchpad.net/~maria-developers >> Post to : maria-developers(a)lists.launchpad.net >> Unsubscribe : https://launchpad.net/~maria-developers >> More help : https://help.launchpad.net/ListHelp >>

2 3

[Maria-developers] Issue with setting up of MariaDB and solve issue MDEV-14896
by Aman Jain 11 Apr '18

11 Apr '18

Hello developers I am setting up the codebase for MariaDB on my system to understand the codebase and solve some issues. But I am facing a problem which I am not able to solve when I run "make" command. Error log: https://pastebin.com/u5fAhMHX OS: MacOS 10.13.3 Python version: 2.7.14 Please let me know how can I set up so that I can run and test the changes. Also, I want to solve issue MDEV-14896. Please connect me to the owner of the issue so that I can discuss the changes. ( https://jira.mariadb.org/browse/MDEV-14896 ) Thanks Aman Jain

3 5

[Maria-developers] 10.3-MDEV-11953
by Alexander Barkov 11 Apr '18

11 Apr '18

Hi Sanja and Igor, In 10.3-MDEV-11953 you're adding a number of main_select_push() / pop_select() around "expr" and other rules that contain "expr". How do you decide when push/pop is needed. Why, for example, LIMIT and ORDER clause don't have their own push/pop. They do create some expressions (Items). What is the rule? Thanks.

1 0

[Maria-developers] FOR UPDATE behavior change in 10.3-MDEV-11953
by Alexander Barkov 11 Apr '18

11 Apr '18

Hi Sanja, I noticed a FOR UPDATE behavior change in 10.3-MDEV-11953 In 10.3 I run this script in a client session: CREATE OR REPLACE TABLE t1 (a INT NOT NULL PRIMARY KEY); CREATE OR REPLACE TABLE t2 (a INT NOT NULL PRIMARY KEY); INSERT INTO t1 VALUES (1); INSERT INTO t2 VALUES (2); BEGIN; SELECT * FROM t1 UNION SELECT * FROM t2 FOR UPDATE; Notice no COMMIT yet! It returns this result: +---+ | a | +---+ | 1 | | 2 | +---+ Now I open a new console, start a new client session and run this script: BEGIN; SELECT * FROM t2 FOR UPDATE; The second session gets locked, as expected. Now I return to the first console and run "COMMIT;". The second console gets unlocked and returns +---+ | a | +---+ | 2 | +---+ Now if I do the same in 10.3-MDEV-11953, the second session does not get locked, it returns the result immediately, even before I typed "COMMIT;" in the first session. Note, if in 10.3-MDEV-11953 I start a transaction with a simpler query (without UNION) in the first session: BEGIN; SELECT * FROM t2 FOR UPDATE; and run the same script in the second session BEGIN; SELECT * FROM t2 FOR UPDATE; then the second session gets locked as expected. Is this behavior change expected? Thanks!

1 0

Re: [Maria-developers] 57673bc: MDEV-14024 PCRE2
by Sergei Golubchik 11 Apr '18

11 Apr '18

Hi, Alexey! On Apr 03, Alexey Botchkov wrote: > revision-id: 57673bcfeecf51d1dd8f4907adc8ca11d84796d3 (mariadb-10.3.5-76-g57673bc) > parent(s): d614d4b2e61552a5b74259f91b3e14b23707743f > committer: Alexey Botchkov > timestamp: 2018-04-03 02:59:51 +0400 > message: > > MDEV-14024 PCRE2. > > server ocde switched to the PCRE2 features. ^^^ s/ocde/code/ > diff --git a/cmake/pcre.cmake b/cmake/pcre.cmake > index 4c11392..271dd06 100644 > --- a/cmake/pcre.cmake > +++ b/cmake/pcre.cmake > @@ -5,24 +5,16 @@ SET(WITH_PCRE "auto" CACHE STRING > > + IF(NOT HAVE_PCRE2 OR WITH_PCRE STREQUAL "bundled") > IF (WITH_PCRE STREQUAL "system") > - MESSAGE(FATAL_ERROR "system pcre is not found or unusable") > + MESSAGE(FATAL_ERROR "system pcre2-8 is not found or unusable") better "pcre2-8 library" or simply "pcre2" > diff --git a/sql/item_cmpfunc.cc b/sql/item_cmpfunc.cc > index 89aa307..08aa293 100644 > --- a/sql/item_cmpfunc.cc > +++ b/sql/item_cmpfunc.cc > @@ -5481,15 +5481,6 @@ int Regexp_processor_pcre::default_regex_flags() > return default_regex_flags_pcre(current_thd); > } > > -void Regexp_processor_pcre::set_recursion_limit(THD *thd) > -{ > - long stack_used; > - DBUG_ASSERT(thd == current_thd); > - stack_used= available_stack_size(thd->thread_stack, &stack_used); > - m_pcre_extra.match_limit_recursion= > - (ulong)((my_thread_stack_size - STACK_MIN_SIZE - stack_used)/my_pcre_frame_size); > -} > - > > /** > Convert string to lib_charset, if needed. > @@ -5537,19 +5528,32 @@ bool Regexp_processor_pcre::compile(String *pattern, bool send_error) > if (!(pattern= convert_if_needed(pattern, &pattern_converter))) > return true; > > - m_pcre= pcre_compile(pattern->c_ptr_safe(), m_library_flags, > - &pcreErrorStr, &pcreErrorOffset, NULL); > + m_pcre= pcre2_compile((PCRE2_SPTR8) pattern->ptr(), pattern->length(), > + m_library_flags, > + &pcreErrorNumber, &pcreErrorOffset, NULL); > > if (m_pcre == NULL) > { > if (send_error) > { > char buff[MAX_FIELD_WIDTH]; > - my_snprintf(buff, sizeof(buff), "%s at offset %d", pcreErrorStr, pcreErrorOffset); > + int lmsg= pcre2_get_error_message(pcreErrorNumber, > + (PCRE2_UCHAR8 *)buff, sizeof(buff)); > + if (lmsg >= 0) > + my_snprintf(buff+lmsg, sizeof(buff)-lmsg, > + " at offset %d", pcreErrorOffset); if lmsg < 0, does it guarantee that the buffer is 0-terminated? > my_error(ER_REGEXP_ERROR, MYF(0), buff); > } > return true; > } > + > + m_pcre_match_data= pcre2_match_data_create_from_pattern(m_pcre, NULL); > + if (m_pcre_match_data == NULL) > + { > + my_error(ER_OUT_OF_RESOURCES, MYF(0)); > + return true; > + } > + > return false; > } > > @@ -5570,124 +5574,43 @@ bool Regexp_processor_pcre::compile(Item *item, bool send_error) > */ > void Regexp_processor_pcre::pcre_exec_warn(int rc) const > { ... > - push_warning_printf(thd, Sql_condition::WARN_LEVEL_WARN, > - ER_REGEXP_ERROR, ER_THD(thd, ER_REGEXP_ERROR), errmsg); > + int errlen= pcre2_get_error_message(rc, buf, sizeof(buf)); > + if (errlen >= 0) > + push_warning_printf(thd, Sql_condition::WARN_LEVEL_WARN, > + ER_REGEXP_ERROR, ER_THD(thd, ER_REGEXP_ERROR), buf); perhaps, issue some generic message if pcre2_get_error_message fails? > } > @@ -5697,14 +5620,12 @@ bool Regexp_processor_pcre::exec(String *str, int offset, > { > if (!(str= convert_if_needed(str, &subject_converter))) > return true; > - m_pcre_exec_rc= pcre_exec_with_warn(m_pcre, &m_pcre_extra, > - str->c_ptr_safe(), str->length(), > - offset, 0, > - m_SubStrVec, array_elements(m_SubStrVec)); > + m_pcre_exec_rc= pcre_exec_with_warn(m_pcre, m_pcre_match_data, > + str->ptr(), str->length(), offset, 0); > if (m_pcre_exec_rc > 0) > { > uint i; > - for (i= 0; i < n_result_offsets_to_convert; i++) > + for (i= 0; i < n_result_offsets_to_convert && i < m_pcre_exec_rc-1; i++) hmm, what does that mean? that both n_result_offsets_to_convert < m_pcre_exec_rc-1 and m_pcre_exec_rc-1 < n_result_offsets_to_convert are possible? > { > /* > Convert byte offset into character offset. > diff --git a/sql/item_cmpfunc.h b/sql/item_cmpfunc.h > index 3d11a22..d3f52ed 100644 > --- a/sql/item_cmpfunc.h > +++ b/sql/item_cmpfunc.h > @@ -25,7 +25,8 @@ > > #include "item_func.h" /* Item_int_func, Item_bool_func */ > #define PCRE_STATIC 1 /* Important on Windows */ > -#include "pcre.h" /* pcre header file */ > +#define PCRE2_CODE_UNIT_WIDTH 8 you sure, you want it here and not, say, in one universally-included headers like my_config.h? You don't define it in other places now, so they include different function names and macros. > +#include "pcre2.h" /* pcre header file */ > #include "item.h" > > extern Item_result item_cmp_type(Item_result a,Item_result b); > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index fe2b9c8..dcf0996 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -111,7 +111,7 @@ > #include "sp_rcontext.h" > #include "sp_cache.h" > #include "sql_reload.h" // reload_acl_and_cache > -#include "pcre.h" > +#include "pcre2.h" do you still need to include pcre header here? > > #ifdef HAVE_POLL_H > #include <poll.h> > diff --git a/storage/mroonga/CMakeLists.txt b/storage/mroonga/CMakeLists.txt > index 5d8e8c1..57d12a1 100644 > --- a/storage/mroonga/CMakeLists.txt > +++ b/storage/mroonga/CMakeLists.txt > @@ -189,8 +189,8 @@ else() > set(MYSQL_VARIANT "MySQL") > endif() > > -if(EXISTS "${MYSQL_SOURCE_DIR}/pcre") > - set(MYSQL_REGEX_INCLUDE_DIR "${MYSQL_SOURCE_DIR}/pcre") > +if(EXISTS "${MYSQL_SOURCE_DIR}/pcre2") > + set(MYSQL_REGEX_INCLUDE_DIR "${MYSQL_SOURCE_DIR}/pcre2/src") eh, is that enough? there were a lot more changes in the server, than just correcting the path. > else() > set(MYSQL_REGEX_INCLUDE_DIR "${MYSQL_SOURCE_DIR}/regex") > endif() Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] FYI: 10.3 master branch head builds OK Launchpad all architectures
by Otto Kekäläinen 11 Apr '18

11 Apr '18

Hello! Just for the record, I've done some small tweaks to 10.3 so that it now builds perfectly and passes build tests on all Launchpad architectures that I have access to. https://launchpad.net/~mysql-ubuntu/+archive/ubuntu/mariadb-10.3/+build s?build_text=&build_state=all Let's hope it stays like this for the rest of the current 10.3 cycle and will not regress by before it is announced GA.. On buildbot the Debian sid and Ubuntu Bionic builders pass now. There are however some sporadic failures left. http://buildbot.askmonty.org/buildbot/grid?branch=10.3&category=main&ca tegory=experimental On Travis everything else passes apart from one test, which has been consistently failing for some time. Everything was green in January when I wrote the protected branches blog post, so this one is a consistently failing regression. https://travis-ci.org/MariaDB/server/builds/362466962 - Otto

2 5

[Maria-developers] Temporary table bug
by Badrul Chowdhury 09 Apr '18

09 Apr '18

Hi, Is anyone actively working on https://jira.mariadb.org/browse/MDEV-12459? If not, I would like to take a look. -- Thanks, Badrul

2 2

[Maria-developers] Test failure overview for Debian/Ubuntu on 10.3 master branch
by Otto Kekäläinen 05 Apr '18

05 Apr '18

(posting this here so to ensure it is recorded in mailing list archive) On Travis the consistently failing tests are: main.type_datetime_hires main.select_pkeycache main.select main.select_jcl6 main.type_float main.func_str main.type_time_hires main.type_timestamp_hires Log: https://api.travis-ci.org/v3/job/362466968/log.txt On buildbot the consistelty failing tests for Debian/Ubuntu are: galera.galera_sst_mysqldump Log: http://buildbot.askmonty.org/buildbot/builders/kvm-deb-jessie-x86/build s/4033/steps/mtr/logs/stdio and http://buildbot.askmonty.org/buildbot/builders/kvm-deb-trusty-ppc64le/b uilds/4546/steps/mtr/logs/stdio Single ones: encryption.innodb_encryption Log: http://buildbot.askmonty.org/buildbot/builders/kvm-deb-sid-x86/builds/6 560/steps/mtr/logs/stdio rocksdb.drop_index_inplace Log: http://buildbot.askmonty.org/buildbot/builders/kvm-deb-trusty-ppc64le/b uilds/4546/steps/mtr/logs/stdio - Otto

1 0

Re: [Maria-developers] Regarding bugs (mysqldump)
by Vicențiu Ciorbaru 01 Apr '18

01 Apr '18

Hi Rutuja! Great analysis again and lets share this with the maria-developers mailing list as well. I've tested your proposed change and the change for dump_all_tables_in_db seems to not be necessary. If you look at how dump_all_tables_in_db gets called, it's followed by dump_all_views_in_db. The actual view creation happens in dump_all_views_in_db, while dump_all_tables_in_db will create some dummy tables for views. We create dummy tables first, so as to avoid having to check dependencies within views. There's more details in the comments of why we do that in the code of get_table_structure: /* Create a table with the same name as the view and with columns of the same name in order to satisfy views that depend on this view. The table will be removed when the actual view is created. The properties of each column, are not preserved in this temporary table, because they are not necessary. This will not be necessary once we can determine dependencies between views and can simply dump them in the appropriate order. */ For dump_selected_tables, your proposed change should be sufficient and according to my tests it works. Please submit a pull request for this. Looking forward to having this fixed soon. :) Unrelated to your analysis: There are plenty of edge cases where views won't work: For example: If we have 2 databases and one view uses something from a different database, if you're not careful about the order of things, it's going to fail. To create a "truly" always working dump one would have to create a full dependency graph and then do a topological sort for all objects in the database that depend on the objects we're dumping. This is outside of the scope of this bug, however it could prove to be a "better way to dump stuff". Regards, Vicentiu On Sun, 1 Apr 2018 at 06:28 Rutuja Surve <rutuja.r.surve(a)gmail.com> wrote: > Hello, > Could you review my following approach regarding the mysqldump bug ( > https://jira.mariadb.org/browse/MDEV-15021) > > I went through the code and observed the following: > 1. dump_all_tables_in_db calls dump_table first (for the list of all > tables), then dump_table calls get_view_structure. > 2. get_view_structure dumps the view for the corresponding table to > mysqldump > 3. After this call, dump_all_tables_in_db calls dump_routines_for_db > > dump_selected_tables also behaves in the exact same way as dump_all_tables_in_db, > following steps 1,2,3, except that it does it for a list of given tables > instead of doing it for all tables. > > We want the routines to get dumped before the views for the dump to become > importable. > There are two solutions for this: > 1. Make dump_all_tables_in_db call dump_routines_for_db before it calls > dump_table. This will dump routines first and views later. > This can be done by shifting the call to dump_routines_for_db before the > loop in which dump_table is called. I.E: > Shift : > if (opt_routines && mysql_get_server_version(mysql) >= 50009) > { > DBUG_PRINT("info", ("Dumping routines for database %s", database)); > *dump_routines_for_db*(database); > } > Before this loop: > while ((table= getTableName(0))) > { > char *end= strmov(afterdot, table); > if (include_table((uchar*) hash_key, end - hash_key)) > { > *dump_table*(table,database); > my_free(order_by); > .... > } > } > > We should do this for both dump_all_tables_in_db and dump_selected_tables. > > 2. Approach 2: > As suggested in the jira thread, we can use negative logic to call > dump_table on tables we intend to skip. > This appears to be a workaround. In the place where dump_all_tables_in_db > is called, we'd rather call dump_selected_tables > with a list of tables that don't use the routines we'll be listing after > that. This would fix the issue but is not a good approach. > > If dump_table doesn't alter the values for opt_routines, the first > approach should fix the issue. > > Thanks, > Rutuja > > > > On Fri, Mar 23, 2018 at 10:36 PM, Rutuja Surve <rutuja.r.surve(a)gmail.com> > wrote: > >> Hello, >> With respect to https://jira.mariadb.org/browse/MDEV-15021 : >> From what I understood from the thread, since the routines are placed >> below the tables/views, they cannot be used in views while importing. >> I went through the problematic_dump.sql file that's attached there. What >> does the '@' in the following mean: >> SET @saved_cs_client = @@character_set_client; >> (I'm not familiar with this syntax) >> Also, what does the following piece of code do: (what does create definer >> mean and does this function simply return 1 as there are no other >> statements after begin? This is a routine defined below test_view) >> CREATE DEFINER=`root`@`localhost` FUNCTION `test_function`() RETURNS >> int(11) >> BEGIN >> >> RETURN 1; >> END ;; >> DELIMITER ; >> From my understanding, test_view uses test_function and test_view2 would >> use test_function2. >> In the mysqldump.c file, I went through the dump_routines_for_db >> function. We'd require to modify that in order to fix the bug. >> >> From the beginner bugs list, I found these 2 bugs that are also related >> to mysql dump that I could take up: >> 1. https://jira.mariadb.org/browse/MDEV-9883 >> (Flag for letting mysqldump support the 'auto' option for character set). >> Where in the code base can I find the file where the ' --default-character-set' >> flag is defined?) >> 2. https://jira.mariadb.org/browse/MDEV-8765 >> The solution for this has already been proposed in the thread. For >> mysqldump to not corrupt 4 byte UTF8 data, "change >> include/my_global.h:#define MYSQL_UNIVERSAL_CLIENT_CHARSET to utf8mb4" >> >> Thanks, >> Rutuja >> > >

1 0

Re: [Maria-developers] d975226a0d6: MDEV-15241: make SIGNAL maximum MESSAGE_TEXT length a larger value
by Sergei Golubchik 01 Apr '18

01 Apr '18

Hi, Varun! On Mar 22, Varun wrote: > revision-id: d975226a0d668775062151c9f30676db3d81c93b (mariadb-10.3.0-642-gd975226a0d6) > parent(s): 85ddd9e8ce11bef089ccc6c86745142d09639c14 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-03-22 12:45:55 +0530 > message: > > MDEV-15241: make SIGNAL maximum MESSAGE_TEXT length a larger value > > Increasing the length of MESSAGE_TEXT from 128 to MYSQL_ERRMSG_SIZE > which is the max length of the error message ok to push. Don't forget to document your change, please. > diff --git a/sql/sql_signal.cc b/sql/sql_signal.cc > index 1da8b145151..13dc3db1f10 100644 > --- a/sql/sql_signal.cc > +++ b/sql/sql_signal.cc > @@ -262,12 +262,12 @@ int Sql_cmd_common_signal::eval_signal_informations(THD *thd, Sql_condition *con > } > /* > Enforce that SET MESSAGE_TEXT = <value> evaluates the value > - as VARCHAR(128) CHARACTER SET UTF8. > + as VARCHAR(MYSQL_ERRMSG_SIZE) CHARACTER SET UTF8. > */ > bool truncated; > String utf8_text; > str= set->val_str(& str_value); > - truncated= assign_fixed_string(thd->mem_root, & my_charset_utf8_bin, 128, > + truncated= assign_fixed_string(thd->mem_root, & my_charset_utf8_bin, MYSQL_ERRMSG_SIZE, > & utf8_text, str); > if (truncated) > { Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] MDEV-14929 - AddressSanitizer: memcpy-param-overlap in Field_longstr::compress
by Sergei Golubchik 31 Mar '18

31 Mar '18

Hi, Sergey, > From 6de643a580b4e706e0201927e43a033f5cb98970 Mon Sep 17 00:00:00 2001 > From: Sergey Vojtovich <svoj(a)mariadb.org> > Date: Wed, 21 Mar 2018 13:54:15 +0400 > Subject: [PATCH] MDEV-14929 - AddressSanitizer: memcpy-param-overlap in > Field_longstr::compress > > Handle overlaping "from" and Field_blob_compressed::value for compressed > blobs similarily to regular blobs. > --- > mysql-test/r/column_compression.result | 13 +++++++++++++ > mysql-test/t/column_compression.test | 12 ++++++++++++ > sql/field.cc | 16 +++++++++++----- > 3 files changed, 36 insertions(+), 5 deletions(-) > > diff --git a/sql/field.cc b/sql/field.cc > index 9da65526e145..1b044ec740e2 100644 > --- a/sql/field.cc > +++ b/sql/field.cc > @@ -8718,17 +8718,23 @@ int Field_blob_compressed::store(const char *from, size_t length, > { > ASSERT_COLUMN_MARKED_FOR_WRITE_OR_COMPUTED; > uint to_length= (uint)MY_MIN(max_data_length(), field_charset->mbmaxlen * length + 1); > + String tmp(from, length, cs); > int rc; > > + if (from >= value.ptr() && from <= value.ptr() + value.length() && > + tmp.copy(from, length, cs)) minor style comment: this could be if (from >= value.ptr() && from <= value.end() && tmp.copy()) Bigger question: Field_blob::store() also checks whether charset conversion is necessary. Why is it not applicable here? And, it'd be good to have a test case where from > value.ptr() (with a substring() function). > + goto oom; > + > if (value.alloc(to_length)) > - { > - set_ptr((uint32) 0, NULL); > - return -1; > - } > + goto oom; > > - rc= compress((char*) value.ptr(), &to_length, from, (uint)length, cs); > + rc= compress((char*) value.ptr(), &to_length, tmp.ptr(), (uint) length, cs); > set_ptr(to_length, (uchar*) value.ptr()); > return rc; > + > +oom: > + set_ptr((uint32) 0, NULL); > + return -1; > } Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 1

[Maria-developers] Patch for pad functions
by jerome brauge 30 Mar '18

30 Mar '18

Hello Alexander, Can you open an MDEV for rpad/lpad (return NULL instead of empty string when sql_mode=oracle) and review this patch ? Regards.

2 1

Re: [Maria-developers] [Commits] c9ae1234814: MDEV-14592: Custom Aggregates Usage Status Variable
by Vicențiu Ciorbaru 26 Mar '18

26 Mar '18

Hi Varun This generally looks good, 2 comments inline. Vicentiu diff --git a/mysql-test/t/custom_aggregate_functions.test b/mysql-test/t/custom_aggregate_functions.test index 20fcc35f39f..a93ec545f01 100644 --- a/mysql-test/t/custom_aggregate_functions.test +++ b/mysql-test/t/custom_aggregate_functions.test @@ -16,7 +16,9 @@ create table t1 (sal int(10),id int(10)); INSERT INTO t1 (sal,id) VALUES (5000,1); INSERT INTO t1 (sal,id) VALUES (2000,1); INSERT INTO t1 (sal,id) VALUES (1000,1); # Please test that this works with subqueries, views, ctes and when calling from # other stored procedures. +show global status like "Feature_custom_aggregate_functions"; select f1(sal) from t1 where id>= 1; +show global status like "Feature_custom_aggregate_functions"; select * from t2; drop table t2; drop function f1; diff --git a/sql/item_sum.cc b/sql/item_sum.cc index d4abdfc614f..7c0390e543f 100644 --- a/sql/item_sum.cc +++ b/sql/item_sum.cc @@ -313,6 +313,8 @@ bool Item_sum::check_sum_func(THD *thd, Item **ref) } } aggr_sel->set_agg_func_used(true); + if (sum_func() == SP_AGGREGATE_FUNC) + aggr_sel->set_custom_agg_func_used(true); update_used_tables(); thd->lex->in_sum_func= in_sum_func; return FALSE; diff --git a/sql/mysqld.cc b/sql/mysqld.cc index c7fbf0a594c..75ccda408b1 100644 --- a/sql/mysqld.cc +++ b/sql/mysqld.cc @@ -8491,6 +8491,7 @@ SHOW_VAR status_vars[]= { {"Executed_triggers", (char*) offsetof(STATUS_VAR, executed_triggers), SHOW_LONG_STATUS}, {"Feature_check_constraint", (char*) &feature_check_constraint, SHOW_LONG }, {"Feature_delay_key_write", (char*) &feature_files_opened_with_delayed_keys, SHOW_LONG }, Please move this up so the variable names are ordered alphabetically. + {"Feature_custom_aggregate_functions", (char*) offsetof(STATUS_VAR, feature_custom_aggregate_functions), SHOW_LONG_STATUS}, {"Feature_dynamic_columns", (char*) offsetof(STATUS_VAR, feature_dynamic_columns), SHOW_LONG_STATUS}, {"Feature_fulltext", (char*) offsetof(STATUS_VAR, feature_fulltext), SHOW_LONG_STATUS}, {"Feature_gis", (char*) offsetof(STATUS_VAR, feature_gis), SHOW_LONG_STATUS}, On Sat, 9 Dec 2017 at 08:53 Varun <varunraiko1803(a)gmail.com> wrote: > revision-id: c9ae12348143f59e875a52e2eb285c5ed7d7af56 > (mariadb-10.3.0-285-gc9ae1234814) > parent(s): c60095a818dce92838940525899a13a05633d148 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2017-12-09 12:22:26 +0530 > message: > > MDEV-14592: Custom Aggregates Usage Status Variable > > --- > mysql-test/r/custom_aggregate_functions.result | 6 ++++++ > mysql-test/t/custom_aggregate_functions.test | 2 ++ > sql/item_sum.cc | 2 ++ > sql/mysqld.cc | 1 + > sql/sql_lex.cc | 2 ++ > sql/sql_lex.h | 3 +++ > sql/sql_select.cc | 2 ++ > 7 files changed, 18 insertions(+) > > diff --git a/mysql-test/r/custom_aggregate_functions.result > b/mysql-test/r/custom_aggregate_functions.result > index ca8612ba652..f70675fa4db 100644 > --- a/mysql-test/r/custom_aggregate_functions.result > +++ b/mysql-test/r/custom_aggregate_functions.result > @@ -11,6 +11,9 @@ create table t1 (sal int(10),id int(10)); > INSERT INTO t1 (sal,id) VALUES (5000,1); > INSERT INTO t1 (sal,id) VALUES (2000,1); > INSERT INTO t1 (sal,id) VALUES (1000,1); > +show global status like "Feature_custom_aggregate_functions"; > +Variable_name Value > +Feature_custom_aggregate_functions 0 > select f1(sal) from t1 where id>= 1; > f1(sal) > 0 > @@ -18,6 +21,9 @@ Warnings: > Note 4093 At line 5 in test.f1 > Note 4093 At line 5 in test.f1 > Note 4093 At line 5 in test.f1 > +show global status like "Feature_custom_aggregate_functions"; > +Variable_name Value > +Feature_custom_aggregate_functions 1 > select * from t2; > sal > 5000 > diff --git a/mysql-test/t/custom_aggregate_functions.test > b/mysql-test/t/custom_aggregate_functions.test > index 20fcc35f39f..a93ec545f01 100644 > --- a/mysql-test/t/custom_aggregate_functions.test > +++ b/mysql-test/t/custom_aggregate_functions.test > @@ -16,7 +16,9 @@ create table t1 (sal int(10),id int(10)); > INSERT INTO t1 (sal,id) VALUES (5000,1); > INSERT INTO t1 (sal,id) VALUES (2000,1); > INSERT INTO t1 (sal,id) VALUES (1000,1); > +show global status like "Feature_custom_aggregate_functions"; > select f1(sal) from t1 where id>= 1; > +show global status like "Feature_custom_aggregate_functions"; > select * from t2; > drop table t2; > drop function f1; > diff --git a/sql/item_sum.cc b/sql/item_sum.cc > index d4abdfc614f..7c0390e543f 100644 > --- a/sql/item_sum.cc > +++ b/sql/item_sum.cc > @@ -313,6 +313,8 @@ bool Item_sum::check_sum_func(THD *thd, Item **ref) > } > } > aggr_sel->set_agg_func_used(true); > + if (sum_func() == SP_AGGREGATE_FUNC) > + aggr_sel->set_custom_agg_func_used(true); > update_used_tables(); > thd->lex->in_sum_func= in_sum_func; > return FALSE; > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index c7fbf0a594c..75ccda408b1 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -8491,6 +8491,7 @@ SHOW_VAR status_vars[]= { > {"Executed_triggers", (char*) offsetof(STATUS_VAR, > executed_triggers), SHOW_LONG_STATUS}, > {"Feature_check_constraint", (char*) &feature_check_constraint, > SHOW_LONG }, > {"Feature_delay_key_write", (char*) > &feature_files_opened_with_delayed_keys, SHOW_LONG }, > + {"Feature_custom_aggregate_functions", (char*) offsetof(STATUS_VAR, > feature_custom_aggregate_functions), SHOW_LONG_STATUS}, > {"Feature_dynamic_columns", (char*) offsetof(STATUS_VAR, > feature_dynamic_columns), SHOW_LONG_STATUS}, > {"Feature_fulltext", (char*) offsetof(STATUS_VAR, > feature_fulltext), SHOW_LONG_STATUS}, > {"Feature_gis", (char*) offsetof(STATUS_VAR, feature_gis), > SHOW_LONG_STATUS}, > diff --git a/sql/sql_lex.cc b/sql/sql_lex.cc > index 6cb065c3985..277469cc0aa 100644 > --- a/sql/sql_lex.cc > +++ b/sql/sql_lex.cc > @@ -2241,6 +2241,7 @@ void st_select_lex::init_query() > select_list_tables= 0; > m_non_agg_field_used= false; > m_agg_func_used= false; > + m_custom_agg_func_used= false; > window_specs.empty(); > window_funcs.empty(); > tvc= 0; > @@ -2278,6 +2279,7 @@ void st_select_lex::init_select() > merged_into= 0; > m_non_agg_field_used= false; > m_agg_func_used= false; > + m_custom_agg_func_used= false; > name_visibility_map= 0; > with_dep= 0; > join= 0; > diff --git a/sql/sql_lex.h b/sql/sql_lex.h > index 069b75628e3..9b40529d3e2 100644 > --- a/sql/sql_lex.h > +++ b/sql/sql_lex.h > @@ -1213,9 +1213,11 @@ class st_select_lex: public st_select_lex_node > */ > bool non_agg_field_used() const { return m_non_agg_field_used; } > bool agg_func_used() const { return m_agg_func_used; } > + bool custom_agg_func_used() const { return m_custom_agg_func_used; } > > void set_non_agg_field_used(bool val) { m_non_agg_field_used= val; } > void set_agg_func_used(bool val) { m_agg_func_used= val; } > + void set_custom_agg_func_used(bool val) { m_custom_agg_func_used= val; } > inline void set_with_clause(With_clause *with_clause); > With_clause *get_with_clause() > { > @@ -1259,6 +1261,7 @@ class st_select_lex: public st_select_lex_node > private: > bool m_non_agg_field_used; > bool m_agg_func_used; > + bool m_custom_agg_func_used; > > /* current index hint kind. used in filling up index_hints */ > enum index_hint_type current_index_hint_type; > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index 96aa227b439..c0d7c720910 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -2909,6 +2909,8 @@ bool JOIN::make_aggr_tables_info() > /* Count that we're using window functions. */ > status_var_increment(thd->status_var.feature_window_functions); > } > + if (select_lex->custom_agg_func_used()) > + > status_var_increment(thd->status_var.feature_custom_aggregate_functions); > > fields= curr_fields_list; > // Reset before execution > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] MDEV 10754 - Patch for trim functions
by jerome brauge 26 Mar '18

26 Mar '18

Hello Alexander, Can you review this patch for trim functions for Oracle ? Regards, Jérôme.

2 2

Re: [Maria-developers] [Commits] 2950feb5c09: MDEV-9959: A serious MariaDB server performance bug
by Sergey Petrunia 26 Mar '18

26 Mar '18

Hi Varun, Please find my feedback below. As disucssed before: please also run the MDEV's test case with this patch and post a comment with the result. On Fri, Mar 23, 2018 at 05:17:57PM +0530, Varun wrote: > revision-id: 2950feb5c09033843a6a32c7431421b45d5812dd (mariadb-10.3.0-646-g2950feb5c09) > parent(s): 9ef7ab221d70630155206910de8a6053db097164 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-03-23 17:17:09 +0530 > message: > > MDEV-9959: A serious MariaDB server performance bug > > step#1: if a derived table has SELECT DISTINCT, provide index statistics for it so that the join optimizer in the > upper select knows that ref access to the table will produce one row. > > Added handling for multiple selects in the derived table > > --- > mysql-test/r/mdev9959.result | 64 ++++++++++++++++++++++++++++++++++++++++++++ > mysql-test/t/mdev9959.test | 28 +++++++++++++++++++ > sql/sql_lex.h | 2 ++ > sql/sql_union.cc | 36 +++++++++++++++++++++++++ > sql/table.cc | 26 +++++++++++++++--- > 5 files changed, 153 insertions(+), 3 deletions(-) > > diff --git a/mysql-test/r/mdev9959.result b/mysql-test/r/mdev9959.result > new file mode 100644 > index 00000000000..5269ba9b734 > --- /dev/null > +++ b/mysql-test/r/mdev9959.result > @@ -0,0 +1,64 @@ > +create table t1(a int); > +insert into t1 values (1),(2),(3),(4),(5),(6); > +create table t2(a int, b int,c int); > +insert into t2(a,b,c) values (1,1,2),(2,2,3),(3,1,4),(4,2,2),(5,1,1),(6,2,5); > +create table t3(a int, b int); > +insert into t3(a,b) values (1,1),(2,2),(2,1),(1,2),(5,1),(9,2); > +analyze select * from t1 , ( (select distinct t2.a from t2 group by c))q where t1.a=q.a; > +id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra > +1 PRIMARY t1 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using where > +1 PRIMARY <derived2> ref key0 key0 5 test.t1.a 1 0.83 100.00 100.00 > +2 DERIVED t2 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using temporary; Using filesort It is not at all clear which parts of the above output matter and which don't, for example, does #rows matter for table t1? Please add a comment (into .test file with --echo # ) describing that table "<derived2>" should have type=ref and rows=1 > +select * from t1 , ( (select distinct t2.a from t2 group by c))q where t1.a=q.a; > +a a > +1 1 > +2 2 > +3 3 > +5 5 > +6 6 > +analyze select * from t1 , ( (select t2.a from t2 group by c))q where t1.a=q.a; > +id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra > +1 PRIMARY t1 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using where > +1 PRIMARY <derived2> ref key0 key0 5 test.t1.a 2 0.83 100.00 100.00 > +2 DERIVED t2 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using temporary; Using filesort > +select * from t1 , ( (select t2.a from t2 group by c))q where t1.a=q.a; > +a a > +1 1 > +2 2 > +3 3 > +5 5 > +6 6 > +analyze select * from t1 , ( (select distinct t2.a from t2 group by c) union all (select distinct t2.a from t2 group by c))q where t1.a=q.a; > +id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra > +1 PRIMARY t1 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using where > +1 PRIMARY <derived2> ref key0 key0 5 test.t1.a 2 1.67 100.00 100.00 > +2 DERIVED t2 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using temporary; Using filesort > +3 UNION t2 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using temporary; Using filesort What does the above test show? We've got type=ref, #rows = 2 for the derived table. The estimate is correct but where does it come from? As far as I understand this patch, it only sets the estimate rows=1 for derived tables. How do we arrive at the value of 2? (If we would have gotten rows=2 without this patch, what's the point of having the testcase as part of this patch?) > +select * from t1 , ( (select distinct t2.a from t2 group by c) union all (select distinct t2.a from t2 group by c))q where t1.a=q.a; > +a a > +1 1 > +1 1 > +2 2 > +2 2 > +3 3 > +3 3 > +5 5 > +5 5 > +6 6 > +6 6 > +analyze select * from t1 , ( (select distinct t2.a from t2 group by c) union (select distinct t2.a from t2 group by c) union (select t3.a from t3 group by b))q where t1.a=q.a; > +id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra > +1 PRIMARY t1 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using where > +1 PRIMARY <derived2> ref key0 key0 5 test.t1.a 2 0.83 100.00 100.00 > +2 DERIVED t2 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using temporary; Using filesort > +3 UNION t2 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using temporary; Using filesort > +4 UNION t3 ALL NULL NULL NULL NULL 6 6.00 100.00 100.00 Using temporary; Using filesort > +NULL UNION RESULT <union2,3,4> ALL NULL NULL NULL NULL NULL 5.00 NULL NULL > +select * from t1 , ( (select distinct t2.a from t2 group by c) union (select distinct t2.a from t2 group by c) union (select t3.a from t3 group by b))q where t1.a=q.a; > +a a > +1 1 > +2 2 > +3 3 > +5 5 > +6 6 > +drop table t1,t2,t3; > diff --git a/mysql-test/t/mdev9959.test b/mysql-test/t/mdev9959.test > new file mode 100644 > index 00000000000..77970f363ac > --- /dev/null > +++ b/mysql-test/t/mdev9959.test > @@ -0,0 +1,28 @@ > +create table t1(a int); > +insert into t1 values (1),(2),(3),(4),(5),(6); > +create table t2(a int, b int,c int); > +insert into t2(a,b,c) values (1,1,2),(2,2,3),(3,1,4),(4,2,2),(5,1,1),(6,2,5); > +create table t3(a int, b int); > +insert into t3(a,b) values (1,1),(2,2),(2,1),(1,2),(5,1),(9,2); > + > +# one select in derived table > + > +#with distinct > +analyze select * from t1 , ( (select distinct t2.a from t2 group by c))q where t1.a=q.a; > +select * from t1 , ( (select distinct t2.a from t2 group by c))q where t1.a=q.a; > +# without distinct > +analyze select * from t1 , ( (select t2.a from t2 group by c))q where t1.a=q.a; > +select * from t1 , ( (select t2.a from t2 group by c))q where t1.a=q.a; > + > +# multiple selects in derived table > + > +#UNION ALL > +analyze select * from t1 , ( (select distinct t2.a from t2 group by c) union all (select distinct t2.a from t2 group by c))q where t1.a=q.a; > +select * from t1 , ( (select distinct t2.a from t2 group by c) union all (select distinct t2.a from t2 group by c))q where t1.a=q.a; > + > +#UNION > +analyze select * from t1 , ( (select distinct t2.a from t2 group by c) union (select distinct t2.a from t2 group by c) union (select t3.a from t3 group by b))q where t1.a=q.a; > +select * from t1 , ( (select distinct t2.a from t2 group by c) union (select distinct t2.a from t2 group by c) union (select t3.a from t3 group by b))q where t1.a=q.a; > + > +drop table t1,t2,t3; > + > diff --git a/sql/sql_lex.h b/sql/sql_lex.h > index f0241a32acf..4128c1dd300 100644 > --- a/sql/sql_lex.h > +++ b/sql/sql_lex.h > @@ -761,6 +761,7 @@ class st_select_lex_unit: public st_select_lex_node { > Procedure *last_procedure; /* Pointer to procedure, if such exists */ > > bool columns_are_renamed; > + bool union_all; /* TRUE if we have UNION ALL operation */ This variable is checked but it is never set? > > void init_query(); > st_select_lex* outer_select(); > @@ -800,6 +801,7 @@ class st_select_lex_unit: public st_select_lex_node { > bool union_needs_tmp_table(); > > void set_unique_exclude(); > + bool check_distinct_in_union(); > > friend struct LEX; > friend int subselect_union_engine::exec(); > diff --git a/sql/sql_union.cc b/sql/sql_union.cc > index 857c9a117f5..43ce0418fe5 100644 > --- a/sql/sql_union.cc > +++ b/sql/sql_union.cc > @@ -1960,3 +1960,39 @@ void st_select_lex_unit::set_unique_exclude() > } > } > } > + > +/* > + Check if the selects in the derived table can give distinct rows I think "can" is the wrong word here. UNION/INTERSECT/EXCEPT output "can" be distinct if the underlying selects produce distinct sets of rows. What we are checking is whether the output will be distinct regardless of what data is there in the queried tables. > + > + for example: > + select * from > + ((select t1.a from t1) op (select t2.a from t2) op (select t3.a from t3)); > + the op here being UNION/INTERSECT/EXCEPT > + > + so this function would check if the derived table like the case above > + can give distinct rows or not. Please describe the criteria and provide a couple of examples (either here or in the comments inside the function). > + > + @retval false Distinct rows are not guarenteed > + @retval true Distinct rows are guanrenteed Typos in the above lines. > + > +*/ > + > +bool st_select_lex_unit::check_distinct_in_union() > +{ > + bool is_intersect_present=FALSE; > + st_select_lex* first= first_select(); > + for(st_select_lex *sl=first; sl ; sl=sl->next_select()) > + is_intersect_present|= (sl->linkage == INTERSECT_TYPE); > + > + if (!union_all) > + return true; > + else > + { > + if (union_select) > + { > + if (!is_intersect_present && !union_select->next_select()) > + return true; > + } > + } > + return false; > +} > diff --git a/sql/table.cc b/sql/table.cc > index f42455a8413..c474d2ed4d7 100644 > --- a/sql/table.cc > +++ b/sql/table.cc > @@ -7113,9 +7113,29 @@ bool TABLE::add_tmp_key(uint key, uint key_parts, > if (!keyinfo->rec_per_key) > return TRUE; > bzero(keyinfo->rec_per_key, sizeof(ulong)*key_parts); > - st_select_lex* lex= pos_in_table_list ? pos_in_table_list->derived->first_select(): NULL; > - if (lex && (lex->options & SELECT_DISTINCT)) > - keyinfo->rec_per_key[key_parts-1]=1; > + > + /* > + For the case when there is a derived table that would give distinct rows, > + the index statistics are passed to the join optimizer to tell that > + a ref access to the derived table will produce only one row. > + */ > + > + st_select_lex_unit* derived= pos_in_table_list ? pos_in_table_list->derived: NULL; > + if (derived) > + { > + /* > + This handles the case when we have a single select in the derived table > + */ > + st_select_lex* first= derived->first_select(); > + if (first && !first->is_part_of_union() && > + first->options & SELECT_DISTINCT) > + keyinfo->rec_per_key[key_parts-1]=1; > + else > + { > + if (check_distinct_in_union()) > + keyinfo->rec_per_key[key_parts-1]=1; > + } I think it would be better if there was a single "if", like so: if ((this is a non-union select with SELECT_DISTINCT || check_distinct_in_union()) { keyinfo->rec_per_key[key_parts-1]=1; } > + } > keyinfo->read_stats= NULL; > keyinfo->collected_stats= NULL; > BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] b718e87caaf: MDEV-11274: Executing EXPLAIN of complex query over join limit causes server to crash
by Sergey Petrunia 26 Mar '18

26 Mar '18

Hi Varun, On Sat, Mar 24, 2018 at 03:31:59AM +0530, Varun wrote: > revision-id: b718e87caaf2d0a60c19cd042dfc04868f4bb0b4 (mariadb-5.5.56-187-gb718e87caaf) > parent(s): dfc79ae949f6d831ff65a08a5d32d14b59ba06e6 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-03-24 03:31:18 +0530 > message: > > MDEV-11274: Executing EXPLAIN of complex query over join limit causes server to crash > > For this case we have a view that is mergeable but we are not able to merge it in the > parent select because that would exceed the maximum tables allowed in the join list, so we > materialise this view > TABLE_LIST::dervied is NULL for such views, it is only set for views which have ALGORITHM=TEMPTABLE > Fixed by making sure TABLE_LIST::derived is set for views that could not be merged > > --- > mysql-test/r/view.result | 197 +++++++++++++++++++++++++++++++++++++++++++++++ > mysql-test/t/view.test | 196 ++++++++++++++++++++++++++++++++++++++++++++++ > sql/table.h | 2 + > 3 files changed, 395 insertions(+) > > diff --git a/sql/table.h b/sql/table.h > index 98f8c7ad73f..7b58f62e659 100644 > --- a/sql/table.h > +++ b/sql/table.h > @@ -46,6 +46,7 @@ struct TABLE_LIST; > class ACL_internal_schema_access; > class ACL_internal_table_access; > class Field; > +struct LEX; > > /* > Used to identify NESTED_JOIN structures within a join (applicable only to What is the above needed for? Please remove this change. Ok to push after this is addressed. > @@ -2125,6 +2126,7 @@ struct TABLE_LIST > DBUG_PRINT("enter", ("Alias: '%s' Unit: %p", > (alias ? alias : "<NULL>"), > get_unit())); > + derived= get_unit(); > derived_type= ((derived_type & (derived ? DTYPE_MASK : DTYPE_VIEW)) | > DTYPE_TABLE | DTYPE_MATERIALIZE); > set_check_materialized(); BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] 108cdbf: MDEV-10871 Add logging capability to pam_user_map.c
by Sergei Golubchik 24 Mar '18

24 Mar '18

Hi, Alexey! On Mar 23, Alexey Botchkov wrote: > > Technically yes, there can be many arguments. > BTW i think some other arguments make sence. > We can specify the configuration file name in arguments (now it's hardcoded > as "/etc/security/user_map.conf"), > then we can specify some 'user: user_map' pairs as arguments, > to make an addition to what is in the user_map.conf or we can go by > withouth the .conf > file at all. > Is it worth adding the todo-mdev? Agree. File name — makes sense, user:map pairs probably not, I don't remember any other pam module doing that. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MDEV-15620 Crash when using "SET @@NEW.a=expr" inside a trigger
by Alexander Barkov 23 Mar '18

23 Mar '18

Hello Sanja, Please review a patch for MDEV-15620. Thanks!

1 0

Re: [Maria-developers] 108cdbf: MDEV-10871 Add logging capability to pam_user_map.c
by Sergei Golubchik 22 Mar '18

22 Mar '18

Hi, Alexey! Looks good, of course. See few minor comments below. On Mar 18, Alexey Botchkov wrote: > revision-id: 108cdbf854df6af1ed2c0fc3a63e1786f6b4b7b4 (mariadb-10.1.31-61-g108cdbf) > parent(s): a0c722d853071e605ea025168da1b01bfe21092c > committer: Alexey Botchkov > timestamp: 2018-03-18 12:03:59 +0400 > message: > > MDEV-10871 Add logging capability to pam_user_map.c. > > 'debug' option added to the pam_user_map.so. > It writes the verbose report to the syslog. don't indent the commit comment, please > > --- > plugin/auth_pam/mapper/pam_user_map.c | 74 +++++++++++++++++++++++++++++++++-- > 1 file changed, 71 insertions(+), 3 deletions(-) > > diff --git a/plugin/auth_pam/mapper/pam_user_map.c b/plugin/auth_pam/mapper/pam_user_map.c > index e62be94..1706630 100644 > --- a/plugin/auth_pam/mapper/pam_user_map.c > +++ b/plugin/auth_pam/mapper/pam_user_map.c > @@ -26,10 +26,13 @@ top: accounting > > #include <stdlib.h> > #include <stdio.h> > +#include <ctype.h> > +#include <string.h> > #include <syslog.h> > #include <grp.h> > #include <pwd.h> > > +#include <security/pam_ext.h> > #include <security/pam_modules.h> > > #define FILENAME "/etc/security/user_map.conf" > @@ -90,9 +93,40 @@ static int user_in_group(const gid_t *user_groups, int ng,const char *group) > } > > > +static void print_groups(pam_handle_t *pamh, const gid_t *user_groups, int ng) > +{ > + char buf[256]; > + char *c_buf= buf, *buf_end= buf+sizeof(buf)-1; > + struct group *gr; > + > + for (; ng > 0; user_groups++, ng--) > + { > + char *c; > + if (!(gr= getgrgid(*user_groups)) || > + !(c= gr->gr_name)) > + continue; > + while (*c) > + { > + if (c_buf == buf_end) > + break; > + *(c_buf++)= *c; > + } > + if (c_buf == buf_end) > + break; > + *(c_buf++)= ','; > + } > + *c_buf= 0; > + pam_syslog(pamh, LOG_DEBUG, "User belongs to groups [%s].\n", buf); s/pam_syslog/SYSLOG_DEBUG/ ? > +} > + > + > +static const char debug_keyword[]= "debug"; > +#define SYSLOG_DEBUG if (mode_debug) pam_syslog > + > int pam_sm_authenticate(pam_handle_t *pamh, int flags, > int argc, const char *argv[]) > { > + int mode_debug= 0; > int pam_err, line= 0; > const char *username; > char buf[256]; > @@ -101,6 +135,14 @@ int pam_sm_authenticate(pam_handle_t *pamh, int flags, > gid_t *groups= group_buffer; > int n_groups= -1; > > + for (; argc > 0; argv++) > + { > + if (strcasecmp(*argv, debug_keyword) == 0) > + mode_debug= 1; > + } can there be many arguments? > + > + SYSLOG_DEBUG(pamh, LOG_DEBUG, "Opening file '%s'.\n", FILENAME); > > + > f= fopen(FILENAME, "r"); > if (f == NULL) > { > @@ -110,12 +152,18 @@ int pam_sm_authenticate(pam_handle_t *pamh, int flags, > > pam_err = pam_get_item(pamh, PAM_USER, (const void**)&username); > if (pam_err != PAM_SUCCESS) > + { > + pam_syslog(pamh, LOG_ERR, "Cannot get username.\n"); > goto ret; > + } > + > + SYSLOG_DEBUG(pamh, LOG_DEBUG, "Incoming username '%s'.\n", username); > > while (fgets(buf, sizeof(buf), f) != NULL) > { > char *s= buf, *from, *to, *end_from, *end_to; > int check_group; > + int cmp_result; > > line++; > > @@ -124,7 +172,11 @@ int pam_sm_authenticate(pam_handle_t *pamh, int flags, > if ((check_group= *s == '@')) > { > if (n_groups < 0) > + { > n_groups= populate_user_groups(username, &groups); > + if (mode_debug) > + print_groups(pamh, groups, n_groups); > + } > s++; > } > from= s; > @@ -139,14 +191,29 @@ int pam_sm_authenticate(pam_handle_t *pamh, int flags, > if (end_to == to) goto syntax_error; > > *end_from= *end_to= 0; > - if (check_group ? > - user_in_group(groups, n_groups, from) : > - (strcmp(username, from) == 0)) > + > + cmp_result= 0; looks redundant ^^^ > + if (check_group) > + { > + cmp_result= user_in_group(groups, n_groups, from); > + SYSLOG_DEBUG(pamh, LOG_DEBUG, "Check if user is in gourp '%s': %s\n", s/gourp/group/ > + from, cmp_result ? "YES":"NO"); > + } > + else > + { > + cmp_result= (strcmp(username, from) == 0); > + SYSLOG_DEBUG(pamh, LOG_DEBUG, "Check if username '%s': %s\n", > + from, cmp_result ? "YES":"NO"); > + } > + if (cmp_result) > { > pam_err= pam_set_item(pamh, PAM_USER, to); > + SYSLOG_DEBUG(pamh, LOG_DEBUG, "User authenticated as '%s'\n", to); may be "user mapped to %s" ? > goto ret; > } > } > + > + SYSLOG_DEBUG(pamh, LOG_DEBUG, "User not found in the list.\n"); > pam_err= PAM_AUTH_ERR; > goto ret; > > @@ -162,6 +229,7 @@ int pam_sm_authenticate(pam_handle_t *pamh, int flags, > return pam_err; > } > > + > int pam_sm_setcred(pam_handle_t *pamh, int flags, > int argc, const char *argv[]) > { Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] GSoC 2018 - Histograms with equal-width bins
by Teodor Niculescu 22 Mar '18

22 Mar '18

Greetings, My name is Teodor Niculescu and I am a second year undergraduate student at the The Faculty of Automatic Control and Computers, University Politehnica of Bucharest, Romania. I am interested in contributing to MariaDB as part of this years Google Summer of Code. I am interested in the project: MDEV-12313 Histograms with equal-width bins. I would like to ask you about a good place to start reading the codebase, in order to help me get an idea about what I need to do for this project. I look forward to hearing from you. Sincerely, Teodor.

2 1

Re: [Maria-developers] MDEV-15492: Subquery crash similar to MDEV-10050
by Sergei Golubchik 21 Mar '18

21 Mar '18

Hi, Oleksandr! On Mar 21, Oleksandr Byelkin wrote: > revision-id: 49a443c53a7f01812299f20e94c7209579efd881 (mariadb-5.5.59-45-g49a443c53a7) > parent(s): 0943b33de3daa0fcbf58803be8e991941de63218 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-03-21 12:13:37 +0100 > message: > > MDEV-15492: Subquery crash similar to MDEV-10050 > > Detection of first execution of PS fixed. > More debug info. > > diff --git a/sql/sql_prepare.cc b/sql/sql_prepare.cc > index a3bf9d6c93c..369a9f3a804 100644 > --- a/sql/sql_prepare.cc > +++ b/sql/sql_prepare.cc > @@ -3819,6 +3819,7 @@ bool Prepared_statement::execute(String *expanded_query, bool open_cursor) > Statement stmt_backup; > Query_arena *old_stmt_arena; > bool error= TRUE; > + bool qc_executed; > > char saved_cur_db_name_buf[SAFE_NAME_LEN+1]; > LEX_STRING saved_cur_db_name= > @@ -3931,12 +3932,14 @@ bool Prepared_statement::execute(String *expanded_query, bool open_cursor) > 1); > error= mysql_execute_command(thd); > MYSQL_QUERY_EXEC_DONE(error); > + qc_executed= FALSE; > } > else > { > thd->lex->sql_command= SQLCOM_SELECT; > status_var_increment(thd->status_var.com_stat[SQLCOM_SELECT]); > thd->update_stats(); > + qc_executed= TRUE; > } > } mostly ok, but this leaves one code path where qc_executed stays uninitialized. Better assign a default value of FALSE when declaring a variable, and set it to TRUE when really executed. ok to push after that. > > @@ -3960,7 +3963,7 @@ bool Prepared_statement::execute(String *expanded_query, bool open_cursor) > thd->set_statement(&stmt_backup); > thd->stmt_arena= old_stmt_arena; > > - if (state == Query_arena::STMT_PREPARED) > + if (state == Query_arena::STMT_PREPARED && !qc_executed) > state= Query_arena::STMT_EXECUTED; > > if (error == 0 && this->lex->sql_command == SQLCOM_CALL) Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] b960f68b102: MDEV-11274: Executing EXPLAIN of complex query over join limit causes server to crash
by Sergey Petrunia 21 Mar '18

21 Mar '18

Hi Varun, On Tue, Jan 23, 2018 at 10:49:37AM +0530, Varun wrote: > revision-id: b960f68b102ec72cf75252b357760e99c5dffc18 (mariadb-5.5.56-144-gb960f68b102) > parent(s): a7a4519a40c58947796c6d9b2e4e58acc18aeef8 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-01-23 10:31:02 +0530 > message: > > MDEV-11274: Executing EXPLAIN of complex query over join limit causes server to crash > > For this case we have a view that is mergeable but we are not able to merge it in the > parent select because that would exceed the maximum tables allowed in the join list, so we > materialise this view > TABLE_LIST::dervied is NULL for such views, it is only set for views which have ALGORITHM=TEMPTABLE > Fixed by using SELECT_LEX_UNIT representing the result of the query within the view definition ... > diff --git a/sql/table.cc b/sql/table.cc > index 9cade76cb78..8f377d5bda8 100644 > --- a/sql/table.cc > +++ b/sql/table.cc > @@ -6817,9 +6817,8 @@ int TABLE_LIST::fetch_number_of_rows() > if (jtbm_subselect) > return 0; > if (is_materialized_derived() && !fill_me) > - > { > - table->file->stats.records= ((select_union*)derived->result)->records; > + table->file->stats.records= ((select_union*)get_unit()->result)->records; > set_if_bigger(table->file->stats.records, 2); So, regular VIEWs with algorithm=TEMPTABLE have derived == view->unit This is set in mysql_make_view(): ... table->derived_type= VIEW_ALGORITHM_TMPTABLE; DBUG_PRINT("info", ("algorithm: TEMPORARY TABLE")); view_select->linkage= DERIVED_TABLE_TYPE; ... However, the view in the example has algorithm=MERGE so the execution doesn't reach that point. The decision to materialized is made in mysql_derived_merge. It calls derived->set_materialized_derived(); which calls set_check_materialized(); I have discussed this with Sanja, and the outcome is that a better solution would be to set TABLE_LIST::derived when we decide that this view is going to be handled with temp.table strategy. set_check_materialized() seems to be the right place to do it (although its name looks confusing. What does one "check"? Probably the name is a leftover from some feature backport from MySQL). Could you change the patch to set TABLE_LIST::derived and see if that works? BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] 0f35480f3d6: MDEV-6707: Wrong result (extra row) with group by, multi-part key
by Sergey Petrunia 20 Mar '18

20 Mar '18

Hi Varun, Nice to see feedback for the first review fixed, but there's more to do. Please find more input below. On Fri, Jan 19, 2018 at 01:07:20AM +0530, Varun wrote: > revision-id: 0f35480f3d60fac91bf1f91f8140ac5f55724139 (mariadb-5.5.56-142-g0f35480f3d6) > parent(s): fafdac3365f4943e73bcefd0e0d07d69997a9724 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-01-18 23:53:35 +0530 > message: > > MDEV-6707: Wrong result (extra row) with group by, multi-part key > > This case involves using a composite key,few parts of which are involved in GROUP BY and few in the MIN/MAX > functions in the select list. > Ranges are created in accordance with the where condition, so during the execution of such queries, > we try to find a prefix using the fields involved in GROUP BY > and we then check if this newly returned prefix lies within the range we had calculated earlier. > > For queries which use composite key, few parts of which are involved in GROUP BY and few in the MIN/MAX functtions > in the select list, we try to find the prefix of the ranges by using the fields involved in the group by clause. > We get extra rows in the output when we have same partial ranges created for the fields in the GROUP BY clause > > This issue can be fixed if we compare such partial ranges and don't lookup if we see the same prefix again. > > --- > mysql-test/r/range.result | 14 +++++++++++++ > mysql-test/r/range_mrr_icp.result | 14 +++++++++++++ > mysql-test/t/range.test | 13 ++++++++++++ > sql/opt_range.cc | 44 ++++++++++++++++++++++++++++++++++++--- > sql/opt_range.h | 5 +++-- > 5 files changed, 85 insertions(+), 5 deletions(-) > > diff --git a/mysql-test/r/range.result b/mysql-test/r/range.result > index 630a692cef6..72b31de4df4 100644 > --- a/mysql-test/r/range.result > +++ b/mysql-test/r/range.result > @@ -2144,3 +2144,17 @@ value1 1000685 12345 > value1 1003560 12345 > value1 1004807 12345 > drop table t1; > +# > +# MDEV-6707: Wrong result (extra row) with group by, multi-part key > +# > +CREATE TABLE t1 (f1 INT, f2 VARCHAR(1), KEY(f2,f1)) ENGINE=InnoDB; > +INSERT INTO t1 VALUES > +(7,'v'),(0,'s'),(9,'l'),(4,'c'); > +explain > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE t1 range f2 f2 9 NULL 2 Using where; Using index for group-by > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +MAX(f1) f2 > +4 c > +DROP TABLE t1; > diff --git a/mysql-test/r/range_mrr_icp.result b/mysql-test/r/range_mrr_icp.result > index 3f5de5b0189..1050bfcd887 100644 > --- a/mysql-test/r/range_mrr_icp.result > +++ b/mysql-test/r/range_mrr_icp.result > @@ -2146,4 +2146,18 @@ value1 1000685 12345 > value1 1003560 12345 > value1 1004807 12345 > drop table t1; > +# > +# MDEV-6707: Wrong result (extra row) with group by, multi-part key > +# > +CREATE TABLE t1 (f1 INT, f2 VARCHAR(1), KEY(f2,f1)) ENGINE=InnoDB; > +INSERT INTO t1 VALUES > +(7,'v'),(0,'s'),(9,'l'),(4,'c'); > +explain > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE t1 range f2 f2 9 NULL 2 Using where; Using index for group-by > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +MAX(f1) f2 > +4 c > +DROP TABLE t1; > set optimizer_switch=@mrr_icp_extra_tmp; > diff --git a/mysql-test/t/range.test b/mysql-test/t/range.test > index 393ca68e945..62e907b7b4a 100644 > --- a/mysql-test/t/range.test > +++ b/mysql-test/t/range.test > @@ -1718,3 +1718,16 @@ where (key1varchar='value1' AND (key2int <=1 OR key2int > 1)); > --echo # The following must show col1=12345 for all rows: > select * from t1; > drop table t1; > + > +--echo # > +--echo # MDEV-6707: Wrong result (extra row) with group by, multi-part key > +--echo # > + > +CREATE TABLE t1 (f1 INT, f2 VARCHAR(1), KEY(f2,f1)) ENGINE=InnoDB; > +INSERT INTO t1 VALUES > +(7,'v'),(0,'s'),(9,'l'),(4,'c'); > + > +explain > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +SELECT MAX(f1), f2 FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2; > +DROP TABLE t1; > diff --git a/sql/opt_range.cc b/sql/opt_range.cc > index 25a9e729a8b..be959764b16 100644 > --- a/sql/opt_range.cc > +++ b/sql/opt_range.cc > @@ -11249,6 +11249,7 @@ int QUICK_RANGE_SELECT::get_next() > @param prefix_length length of cur_prefix > @param group_key_parts The number of key parts in the group prefix > @param cur_prefix prefix of a key to be searched for > + @param save_last_range INOUT Saving the last range we encountered > > Each subsequent call to the method retrieves the first record that has a > prefix with length prefix_length and which is different from cur_prefix, > @@ -11271,7 +11272,8 @@ int QUICK_RANGE_SELECT::get_next() > > int QUICK_RANGE_SELECT::get_next_prefix(uint prefix_length, > uint group_key_parts, > - uchar *cur_prefix) > + uchar *cur_prefix, > + QUICK_RANGE **save_last_range) > { > DBUG_ENTER("QUICK_RANGE_SELECT::get_next_prefix"); > const key_part_map keypart_map= make_prev_keypart_map(group_key_parts); > @@ -11301,8 +11303,43 @@ int QUICK_RANGE_SELECT::get_next_prefix(uint prefix_length, > last_range= 0; > DBUG_RETURN(HA_ERR_END_OF_FILE); > } > + > last_range= *(cur_range++); > > + /* > + While calculating these prefixes we might encounter a case where there > + woud be same partial ranges for multiple ranges. Typo. > + An example would be > + select max(f1), f2 from t1 where f2 ='c' AND f1 <> 9 group by f2; > + so the ranges would be > + (NULL,c <= f1,f2 <= c,9) > + (c,9 <= f1,f2 <= c,+infinity) This looks as if index was defined on (f1, f2) ? Then the comment doesn't make sense because loose index scan does not support queries in form: select max(key_part1) ... group by key_part2 It only can handle them when the index is on (f2,f1), but this is counterintuitive for the reader that is not familiar with the testcase for this bug. Please rephrase the comment. Please use key_part1 and key_part2 as column names. Tuple comparisons should look like this (note the braces): (a,b) < (c,d) . > + > + In this case for calculating prefixes with the group by field we take up the > + partial ranges involving field f2 those would be > + c<= f2 <=c > + c<= f2 <=c > + > + So we lookup rows with the same prefix in all such ranges and > + then we check for the other part(in this case f1) in ALL the ranges. > + So if a record lies in a range, then it would satisfy both the partial > + ranges in this case and therefore there would be multiple outputs for > + the same row. > + For such cases we should calculate the prefix only when we have the next > + partial range different from the previous one. > + */ > + > + > + if (*save_last_range) > + { > + if (!key_tuple_cmp(key_part_info, (*save_last_range)->min_key, > + last_range->min_key, prefix_length)) > + { > + last_range=NULL; > + continue; > + } > + } > + *save_last_range= last_range; > key_range start_key, end_key; > last_range->make_min_endpoint(&start_key, prefix_length, keypart_map); > last_range->make_max_endpoint(&end_key, prefix_length, keypart_map); > @@ -13315,7 +13352,7 @@ QUICK_GROUP_MIN_MAX_SELECT(TABLE *table, JOIN *join_arg, bool have_min_arg, > seen_first_key(FALSE), doing_key_read(FALSE), min_max_arg_part(min_max_arg_part_arg), > key_infix(key_infix_arg), key_infix_len(key_infix_len_arg), > min_functions_it(NULL), max_functions_it(NULL), > - is_index_scan(is_index_scan_arg) > + is_index_scan(is_index_scan_arg), save_last_range(NULL) This is not the right place for this initialization. It needs to be in QUICK_GROUP_MIN_MAX_SELECT::reset(), as the quick select may be executed multiple times. Testcase: create table ten(a int); insert into ten values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); MariaDB [test]> select (SELECT MAX(f1) as MAXVAL FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2) AS SUBQ from ten; +------+ | SUBQ | +------+ | 4 | | 4 | | 4 | | 4 | | 4 | | 4 | | 4 | | 4 | | 4 | | 4 | +------+ 10 rows in set (0.00 sec) Now, let's add a HAVING clause (which always evaluates to TRUE but forces the subquery to be recomputed: MariaDB [test]> select (SELECT MAX(f1) as MAXVAL FROM t1 WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2 having maxval<100+ten.a) AS SUBQ from ten; +------+ | SUBQ | +------+ | 4 | | NULL | | NULL | | NULL | | NULL | | NULL | | NULL | | NULL | | NULL | | NULL | +------+ 10 rows in set (0.00 sec) To make sure this is loose index, try with IGNORE INDEX: MariaDB [test]> select (SELECT MAX(f1) as MAXVAL FROM t1 ignore index(f2) WHERE f2 LIKE 'c%' AND f1 <> 9 GROUP BY f2 having maxval<100+ten.a) AS SUBQ from ten; +------+ | SUBQ | +------+ | 4 | | 4 | | 4 | | 4 | | 4 | | 4 | | 4 | | 4 | | 4 | | 4 | +------+ 10 rows in set (0.00 sec) Please fix this and add the above into the testcase. > { > head= table; > index= use_index; > @@ -13968,7 +14005,8 @@ int QUICK_GROUP_MIN_MAX_SELECT::next_prefix() > uchar *cur_prefix= seen_first_key ? group_prefix : NULL; > if ((result= quick_prefix_select->get_next_prefix(group_prefix_len, > group_key_parts, > - cur_prefix))) > + cur_prefix, > + &save_last_range))) > DBUG_RETURN(result); > seen_first_key= TRUE; > } > diff --git a/sql/opt_range.h b/sql/opt_range.h > index b8b46ae5ab1..206d65f6da4 100644 > --- a/sql/opt_range.h > +++ b/sql/opt_range.h > @@ -477,7 +477,7 @@ class QUICK_RANGE_SELECT : public QUICK_SELECT_I > int get_next(); > void range_end(); > int get_next_prefix(uint prefix_length, uint group_key_parts, > - uchar *cur_prefix); > + uchar *cur_prefix, QUICK_RANGE **save_last_range); > bool reverse_sorted() { return 0; } > bool unique_key_range(); > int init_ror_merged_scan(bool reuse_handler, MEM_ROOT *alloc); > @@ -916,7 +916,8 @@ class QUICK_GROUP_MIN_MAX_SELECT : public QUICK_SELECT_I > Use index scan to get the next different key instead of jumping into it > through index read > */ > - bool is_index_scan; > + bool is_index_scan; > + QUICK_RANGE *save_last_range; Please add a comment describing the new member variable. > public: > /* > The following two members are public to allow easy access from BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] GSoC 2018 Student Self Introduction & Question
by zhzhzoo zhang 20 Mar '18

20 Mar '18

Hello everybody, I am a first year graduate in Peking University major in math. I started to take part in algorithm contests at grade 7 so I have a lot of programming experience and I am very well trained in algorithms and data structures. In my childhood memory MySQL was the most popular DBMS choice for personal website (LAMP stack). However it was acquired by Oracle as I grew up. In this year's GSoC projects, `optimizer trace' interested me particularly. It would be great for MariaDB to have this functionality. This functionality is already implemented in MySQL. So my question is, can I read MySQL's source code implementing this function? Would that cause legal problems? Thanks very much! Regards, Zhang Shuai.

3 2

Re: [Maria-developers] [Commits] a99dcdf7e9e: MDEV-6731: Valgrind warnings 'Invalid read' in subselect_engine::calc_const_tables with SQ in WHERE and
by Sergey Petrunia 16 Mar '18

16 Mar '18

Hi Varun, Please fix the MDEV# in the commit comment. Ok to push after that. On Wed, Mar 14, 2018 at 12:11:20PM +0530, Varun wrote: > revision-id: a99dcdf7e9ede3b96466eb1294284141805747fc (mariadb-10.0.30-308-ga99dcdf7e9e) > parent(s): 48c11d407b409a1291c84782518c69ee9138ce72 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-03-14 12:09:49 +0530 > message: > > MDEV-6731: Valgrind warnings 'Invalid read' in subselect_engine::calc_const_tables with SQ in WHERE and > HAVING, ORDER BY, materialization+semijoin > > During cleanup a pointer to the materialised table that was freed was not set to NULL > > --- > mysql-test/r/having.result | 14 ++++++++++++++ > mysql-test/t/having.test | 18 ++++++++++++++++++ > sql/sql_select.cc | 4 +++- > 3 files changed, 35 insertions(+), 1 deletion(-) > > diff --git a/mysql-test/r/having.result b/mysql-test/r/having.result > index 0b933427303..99059c25fbd 100644 > --- a/mysql-test/r/having.result > +++ b/mysql-test/r/having.result > @@ -723,4 +723,18 @@ SELECT * FROM t1 JOIN t2 ON c1 = c2 HAVING c2 > 'a' ORDER BY c2 LIMIT 1; > c1 c2 > x x > DROP TABLE t1,t2; > +# > +# MDEV-6736: Valgrind warnings 'Invalid read' in subselect_engine::calc_const_tables with SQ > +# in WHERE and HAVING, ORDER BY, materialization+semijoin > +# > +CREATE TABLE t1 (a INT) ENGINE=MyISAM; > +INSERT INTO t1 VALUES (3),(8); > +CREATE TABLE t2 (b INT) ENGINE=MyISAM; > +INSERT INTO t2 VALUES (2),(1); > +SELECT a FROM t1 > +WHERE 9 IN ( SELECT MIN( a ) FROM t1 ) > +HAVING a <> ( SELECT COUNT(*) FROM t2 ) > +ORDER BY a; > +a > +DROP TABLE t1,t2; > End of 10.0 tests > diff --git a/mysql-test/t/having.test b/mysql-test/t/having.test > index 1682fe5b874..e624cd630dd 100644 > --- a/mysql-test/t/having.test > +++ b/mysql-test/t/having.test > @@ -759,4 +759,22 @@ SELECT * FROM t1 JOIN t2 ON c1 = c2 HAVING c2 > 'a' ORDER BY c2 LIMIT 1; > > DROP TABLE t1,t2; > > +--echo # > +--echo # MDEV-6736: Valgrind warnings 'Invalid read' in subselect_engine::calc_const_tables with SQ > +--echo # in WHERE and HAVING, ORDER BY, materialization+semijoin > +--echo # > + > +CREATE TABLE t1 (a INT) ENGINE=MyISAM; > +INSERT INTO t1 VALUES (3),(8); > + > +CREATE TABLE t2 (b INT) ENGINE=MyISAM; > +INSERT INTO t2 VALUES (2),(1); > + > +SELECT a FROM t1 > +WHERE 9 IN ( SELECT MIN( a ) FROM t1 ) > +HAVING a <> ( SELECT COUNT(*) FROM t2 ) > +ORDER BY a; > + > +DROP TABLE t1,t2; > + > --echo End of 10.0 tests > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index fd8ff6eb016..37d68c730dd 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -11476,13 +11476,15 @@ void JOIN_TAB::cleanup() > } > else > { > + TABLE_LIST *tmp= table->pos_in_table_list; > end_read_record(&read_record); > - table->pos_in_table_list->jtbm_subselect->cleanup(); > + tmp->jtbm_subselect->cleanup(); > /* > The above call freed the materializedd temptable. Set it to NULL so > that we don't attempt to touch it if JOIN_TAB::cleanup() is invoked > multiple times (it may be) > */ > + tmp->table= NULL; > table=NULL; > } > DBUG_VOID_RETURN; > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] d8e3cfacebc: MDEV-14616: WSREP has not yet prepared node for application use error
by Sergei Golubchik 15 Mar '18

15 Mar '18

Hi, Jan! On Feb 13, jan wrote: > revision-id: d8e3cfacebc47849532ef3e7bbf18f9da53bab88 (mariadb-10.1.31-20-gd8e3cfacebc) > parent(s): 474f3df092cc73342b2da03aa03ba5c66930f596 > author: Jan Lindström > committer: Jan Lindström > timestamp: 2018-02-13 10:04:48 +0200 > message: > > MDEV-14616: WSREP has not yet prepared node for application use error > > Contains following Galera fixes: > > MW-405 Make sure wsrep is ready in wait_until_connected_again.inc What is "MW-405", can I see the original bug report somewhere? > wait_until_connected_again issues 'SHOW STATUS' query repeatedly > until mysqld replies without errors. However, SHOW STATUS is > treated specially by wsrep in that it is allowed to proceed > even if wsrep is not yet in ready state. As a consequence, > after returning from wait_until_connected_again, wsrep may > not be ready yet and subsequent queries may fail with error > "1047 WSREP has not yet prepared node for application use". > To avoid those errors, the patch includes > wait_wsrep_ready.inc at the end of the wait_until_connected_again.inc I don't quite understand the issue. Old code did wait_wsrep_ready.inc after wait_until_connected_again.inc. Now wait_wsrep_ready.inc is included in wait_until_connected_again.inc. This seems to imply that the bug was "there were some cases where wait_wsrep_ready.inc was not included after wait_until_connected_again.inc". There are, of course, two ways to fix such a bug. Either include wait_wsrep_ready.inc after wait_until_connected_again.inc, or include wait_wsrep_ready.inc in wait_until_connected_again.inc. The latter fix implies that there are no possible cases when one would want to wait for a server to be connected, but before wsrep is ready. Are there? For the purpose of testing, that could be useful, could it not? > kill_galera.inc can no longer rely on wait_until_connected_again.inc. > This is because wait_until_connected_again now tries to make sure > that the server it is connected eventually transition to ready state. > Whereas some tests may need to kill galera while the server is in a > non-primary view. Which means that there are cases when a test needs the server connected, but before wsrep is ready, right? > MW-408 Fix 'WSREP error while trying to determine node state' > > mysql-test-run.pl sporadically reports 'WSREP error while > trying to determine node state' right after starting servers > for test execution. This happens because we try to execute a > SELECT statement that queries the current value of status variable > wsrep_ready. If this statement fails, the above message is reported. > The failure is due to fact that wsrep may return error > ER_LOCK_WAIT_TIMEOUT (on any SELECT statement) if it is not ready > and wsrep_sync_wait enabled for SELECTs. The fix is to disable > wsrep_sync_wait for the session that issues those SELECT statements. 1. Could you put this fix in a separate commit please? 2. Sorry, I don't understand. Where do "we try to execute a SELECT statement that queries the current value of status variable wsrep_ready" ? > --- > mysql-test/include/restart_mysqld.inc | 3 - > mysql-test/include/start_mysqld.inc | 3 - > mysql-test/include/wait_until_connected_again.inc | 11 ++- > mysql-test/mysql-test-run.pl | 104 ++++++++++++++++++++++ > mysql-test/suite/galera/include/kill_galera.inc | 1 - > 5 files changed, 111 insertions(+), 11 deletions(-) > > diff --git a/mysql-test/include/restart_mysqld.inc b/mysql-test/include/restart_mysqld.inc > index a0447280ff5..940e081c431 100644 > --- a/mysql-test/include/restart_mysqld.inc > +++ b/mysql-test/include/restart_mysqld.inc > @@ -50,9 +50,6 @@ if (!$restart_parameters) > # Call script that will poll the server waiting for it to be back online again > --source include/wait_until_connected_again.inc > > -# Wait for wsrep > ---source include/wait_wsrep_ready.inc > - > # Turn off reconnect again > --disable_reconnect > > diff --git a/mysql-test/include/start_mysqld.inc b/mysql-test/include/start_mysqld.inc > index 04dff714d49..e31f26aad8c 100644 > --- a/mysql-test/include/start_mysqld.inc > +++ b/mysql-test/include/start_mysqld.inc > @@ -16,9 +16,6 @@ if (!$restart_parameters) > # Call script that will poll the server waiting for it to be back online again > --source include/wait_until_connected_again.inc > > -# Wait for wsrep > ---source include/wait_wsrep_ready.inc > - > # Turn off reconnect again > --disable_reconnect > > diff --git a/mysql-test/include/wait_until_connected_again.inc b/mysql-test/include/wait_until_connected_again.inc > index 6f64ef45440..4958d276ebd 100644 > --- a/mysql-test/include/wait_until_connected_again.inc > +++ b/mysql-test/include/wait_until_connected_again.inc > @@ -11,10 +11,7 @@ let $counter= 5000; > let $mysql_errno= 9999; > while ($mysql_errno) > { > - # Strangely enough, the server might return "Too many connections" > - # while being shutdown, thus 1040 is an "allowed" error > - # See BUG#36228 > - --error 0,1040,1053,2002,2003,2005,2006,2013,1927 > + --error 0,ER_SERVER_SHUTDOWN,ER_CONNECTION_KILLED,2002,2003,2006,2013 > show status; > > dec $counter; > @@ -26,3 +23,9 @@ while ($mysql_errno) > } > --enable_query_log > --enable_result_log > + > +# WSREP: SHOW STATUS queries are allowed even if wsrep > +# is not ready. Make sure wsrep is ready before > +# returning from this script > + > +--source include/wait_wsrep_ready.inc > diff --git a/mysql-test/mysql-test-run.pl b/mysql-test/mysql-test-run.pl > index eaec51b82b4..324f2e98ebd 100755 > --- a/mysql-test/mysql-test-run.pl > +++ b/mysql-test/mysql-test-run.pl > @@ -2883,6 +2883,49 @@ sub mysql_server_wait { > $warn_seconds); > } > > +sub have_wsrep() { > + my $wsrep_on= $mysqld_variables{'wsrep-on'}; > + return defined $wsrep_on > +} > + > + > +sub check_wsrep_support() { > + if (have_wsrep()) > + { > + mtr_report(" - binaries built with wsrep patch"); > + > + # ADD scripts to $PATH to that wsrep_sst_* can be found > + my ($path) = grep { -f "$_/wsrep_sst_rsync"; } "$::bindir/scripts", $::path_client_bindir; > + mtr_error("No SST scripts") unless $path; > + $ENV{PATH}="$path:$ENV{PATH}"; > + > + # Check whether WSREP_PROVIDER environment variable is set. > + if (defined $ENV{'WSREP_PROVIDER'}) { > + if ((mtr_file_exists($ENV{'WSREP_PROVIDER'}) eq "") && > + ($ENV{'WSREP_PROVIDER'} ne "none")) { > + mtr_error("WSREP_PROVIDER env set to an invalid path"); > + } > + # WSREP_PROVIDER is valid; set to a valid path or "none"). > + mtr_verbose("WSREP_PROVIDER env set to $ENV{'WSREP_PROVIDER'}"); > + } else { > + # WSREP_PROVIDER env not defined. Lets try to locate the wsrep provider > + # library. > + my $file_wsrep_provider= > + mtr_file_exists("/usr/lib/galera/libgalera_smm.so", > + "/usr/lib64/galera/libgalera_smm.so"); > + > + if ($file_wsrep_provider ne "") { > + # wsrep provider library found ! > + mtr_verbose("wsrep provider library found : $file_wsrep_provider"); > + $ENV{'WSREP_PROVIDER'}= $file_wsrep_provider; > + } else { > + mtr_verbose("Could not find wsrep provider library, setting it to 'none'"); > + $ENV{'WSREP_PROVIDER'}= "none"; > + } > + } > + } > +} > + > sub create_config_file_for_extern { > my %opts= > ( > @@ -3341,6 +3384,62 @@ sub run_query { > return $res > } > > +sub run_query_output { > + my ($mysqld, $query, $outfile)= @_; > + > + my $args; > + mtr_init_args(\$args); > + mtr_add_arg($args, "--defaults-file=%s", $path_config_file); > + mtr_add_arg($args, "--defaults-group-suffix=%s", $mysqld->after('mysqld')); > + > + mtr_add_arg($args, "--silent"); > + mtr_add_arg($args, "--execute=%s", $query); > + > + my $res= My::SafeProcess->run > + ( > + name => "run_query_output -> ".$mysqld->name(), > + path => $exe_mysql, > + args => \$args, > + output => $outfile, > + error => $outfile > + ); > + > + return $res > +} > + > +sub wait_wsrep_ready($$) { > + my ($tinfo, $mysqld)= @_; > + > + my $sleeptime= 100; # Milliseconds > + my $loops= ($opt_start_timeout * 1000) / $sleeptime; > + > + my $name= $mysqld->name(); > + my $outfile= "$opt_vardir/tmp/$name.wsrep_ready"; > + my $query= "SET SESSION wsrep_sync_wait = 0; > + SELECT VARIABLE_VALUE > + FROM INFORMATION_SCHEMA.GLOBAL_STATUS > + WHERE VARIABLE_NAME = 'wsrep_ready'"; > + > + for (my $loop= 1; $loop <= $loops; $loop++) > + { > + if (run_query_output($mysqld, $query, $outfile) != 0) > + { > + $tinfo->{logfile}= "WSREP error while trying to determine node state"; > + return 0; > + } > + > + if (mtr_grab_file($outfile) =~ /^ON/) > + { > + unlink($outfile); > + return 1; > + } > + > + mtr_milli_sleep($sleeptime); > + } > + > + $tinfo->{logfile}= "WSREP did not transition to state READY"; > + return 0; > +} > > sub do_before_run_mysqltest($) > { > @@ -5380,6 +5479,11 @@ sub start_servers($) { > $tinfo->{comment}= "Failed to start ".$_->name() . "\n"; > return 1; > } > + > + if (have_wsrep() && !wait_wsrep_ready($tinfo, $_)) > + { > + return 1; > + } > } > return 0; > } > diff --git a/mysql-test/suite/galera/include/kill_galera.inc b/mysql-test/suite/galera/include/kill_galera.inc > index c61bad8e19d..f95dccf2185 100644 > --- a/mysql-test/suite/galera/include/kill_galera.inc > +++ b/mysql-test/suite/galera/include/kill_galera.inc > @@ -18,4 +18,3 @@ > exit(0); > EOF > > ---source include/wait_until_disconnected.inc > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] dc3d6c9edb3: MDEV-15477: SESSION_SYSVARS_TRACKER does not track last_gtid
by Sergei Golubchik 09 Mar '18

09 Mar '18

Hi, Oleksandr! On Mar 09, Oleksandr Byelkin wrote: > revision-id: dc3d6c9edb3536e5f0c87ac1e9c04891d50fc420 (mariadb-10.2.13-32-gdc3d6c9edb3) > parent(s): 6cf25e5eb76923550173a0059036d80c80a053d6 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-03-09 14:39:40 +0100 > message: > > MDEV-15477: SESSION_SYSVARS_TRACKER does not track last_gtid > > register changes of last_gtid > > diff --git a/mysql-test/t/session_tracker_last_gtid.test b/mysql-test/t/session_tracker_last_gtid.test > --- /dev/null > +++ b/mysql-test/t/session_tracker_last_gtid.test > @@ -0,0 +1,18 @@ > + > +--source include/have_innodb.inc > +--source include/have_binlog_format_mixed_or_statement.inc Why "mixed or statement"? > diff --git a/sql/log.cc b/sql/log.cc > --- a/sql/log.cc > +++ b/sql/log.cc > @@ -5945,7 +5945,20 @@ MYSQL_BIN_LOG::write_gtid_event(THD *thd, bool standalone, > } > if (err) > DBUG_RETURN(true); > + bool changed_gtid= (thd->last_commit_gtid.seq_no != gtid.seq_no); > thd->last_commit_gtid= gtid; > + if (changed_gtid && > + thd->session_tracker.get_tracker(SESSION_SYSVARS_TRACKER)->is_enabled()) > + { > + sys_var *svar; > + mysql_mutex_lock(&LOCK_plugin); > + if ((svar= find_sys_var_ex(thd, "last_gtid", > + sizeof("last_gtid") - 1, > + false, true))) > + thd->session_tracker.get_tracker(SESSION_SYSVARS_TRACKER)-> > + mark_as_changed(thd, (LEX_CSTRING*)svar); > + mysql_mutex_unlock(&LOCK_plugin); > + } No, please. You svar will always be Sys_last_gtid. There is no need to search for it on every write_gtid_event(), under a mutex. See in sys_vars.cc around Sys_autocommit how to do it easier. Also, perhaps it'd be safer to rename last_commit_gtid to m_last_commit_gtid, make it private and provide a setter and a getter. The setter will update the tracker. I'll make sure that last_commit_gtid is not changed anywhere else bypassing the tracker. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 6cf25e5eb76: Session tracking info support in mysqltest
by Sergei Golubchik 09 Mar '18

09 Mar '18

Hi, Oleksandr! On Mar 09, Oleksandr Byelkin wrote: > revision-id: 6cf25e5eb76923550173a0059036d80c80a053d6 (mariadb-10.2.13-31-g6cf25e5eb76) > parent(s): f2b91c878cf4ec53c292caba7f5907d1ef724de4 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-03-09 14:27:13 +0100 > message: > > Session tracking info support in mysqltest Add a test case for it. Don't forget to test whether ONCE modifier works. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] f2b91c878cf: Reset connection support in mysqltest
by Sergei Golubchik 09 Mar '18

09 Mar '18

Hi, Oleksandr! On Mar 09, Oleksandr Byelkin wrote: > revision-id: f2b91c878cf4ec53c292caba7f5907d1ef724de4 (mariadb-10.2.13-30-gf2b91c878cf) > parent(s): 8f98835bb86550a68a6c412d8417cc169dfdc206 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-03-09 14:26:10 +0100 > message: > > Reset connection support in mysqltest You add a feature - add a test case for it too. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MariaDB 5.5, Jemalloc, TokuDB, RHEL
by Michal Schorm 08 Mar '18

08 Mar '18

Hello, so far we packed MariaDB 5.5 for RHEL 7 without TokuDB and without Jemalloc. Recently, a customer asked about the availability of TokuDB, so I'm doing a research, if its possible to pack it without any issues for the users updating. I have a several questions: 1) Is Jemmaloc used elsewhere from TokuDB ? 2) If yes, what all would be affected? Could any issues arise during update from build without Jemalloc to build with it? Any expected performance changes? 3) Would be the use of the bundled Jemalloc (on of the) best option(s)? 4) Anything else that I missed? -- Michal Schorm Associate Software Engineer Core Services - Databases Team Red Hat

2 1

[Maria-developers] Introduction as GSoC 2018 aspirant
by Arshdeep Singh Sachdeva 05 Mar '18

05 Mar '18

Dear MariaDB Developers, I am Arshdeep Singh Sachdeva, currently studying BTech ( C.S.E. ) from Punjab Engineering College, Chandigarh, India. I am interested in contibuting to MariaDB and participating in GSoC 2018 under your guidance. I am proficient in C++, python, linux, mysql. I have already used MariaDB in my projects in college. Also I am presently going through documentation and tutorials. Thanks.

1 0

[Maria-developers] Introduction as a GSoC aspirant
by Aman Jain 05 Mar '18

05 Mar '18

Hello developers and mentors I am Aman and I am interested in contributing to MariaDB as a GSoC'18 aspirant. I am familiar with C++, Web development, SQL, MongoDB and shell scripting and I have set up the MariaDB server. Please suggest me where I can find the source code so that I can get a better understanding of the project ideas and solve some issues to get started. Thanks Aman Jain

2 1

Re: [Maria-developers] [Commit] MDEV-12746 out-of-order retry
by Kristian Nielsen 17 Feb '18

17 Feb '18

andrei.elkin(a)pp.inet.fi writes: > commit 3cebb54e6387a7eace1757c82ed0efd6e11590b9 > Author: Andrei Elkin <andrei.elkin(a)mariadb.com> > Date: Fri Feb 9 15:00:23 2018 +0200 > > MDEV-12746 rpl.rpl_parallel_optimistic_nobinlog fails committing > out of order at retry > > The 2nd issue was out of order commit by transactions that > are ordered after an erroring out transaction. The latter was possible > in the test thanks to the above retry failure. > Out-of-order committing was done despite the slave was stopping. Let me see if I understand this correctly: Normally, a transaction in optimistic parallel replication cannot commit out of order because it either waits for the prior transaction to commit first, or, if the prior transaction has stopped with an error earlier, then the following transaction will not even start. But in this case, during retry, there was missing the check that the prior transaction stopped with an error. Therefore, the following transaction's wait_for_prior_commit() does nothing, and it wrongly goes ahead and runs and even commits, even though an earlier transaction failed and replication is stopping. Correct? That's a nice catch of a subtle, but potentially serious bug. I wonder how this is possible though. As I recall, when a transaction fails, this failure is marked in the wait_for_prior_commit data structures. So the retrying transaction should have gotten an error in wait_for_prior_commit() and aborted the retries. So why does this not happen in this case? > The test failures were of two sorts. One is that the number of potential > temporary errors actually exceeds the default value of the slave retry > option. > Increased @@global.slave_transaction_retries fixes that part. Hm, this shouldn't be needed, I think. The retry waits for all prior transactions to commit first, so after the first retry no further retries should be caused by other parallel replication workers. So I would expect this problem to be a side-effect of the other, real issue, and this increase should not be needed (and might hide another problem later, if left in). > diff --git a/sql/rpl_parallel.cc b/sql/rpl_parallel.cc > index 34f9113f7fe..9f1d23a1161 100644 > --- a/sql/rpl_parallel.cc > +++ b/sql/rpl_parallel.cc > @@ -745,7 +761,25 @@ retry_event_group(rpl_group_info *rgi, rpl_parallel_thread *rpt, > for (;;) > { > mysql_mutex_lock(&entry->LOCK_parallel_entry); > - register_wait_for_prior_event_group_commit(rgi, entry); > + if (entry->stop_on_error_sub_id == (uint64) ULONGLONG_MAX || > +#ifndef DBUG_OFF > + (DBUG_EVALUATE_IF("simulate_mdev_12746", 1, 0)) || > +#endif > + rgi->gtid_sub_id < entry->stop_on_error_sub_id) > + { > + register_wait_for_prior_event_group_commit(rgi, entry); > + } > + else > + { > + /* > + An earlier transaction by another worker may have not had not > + have the current one in subsequent committers list and thus > + the actual induced error status could not have been sumbitted. > + So it should be set by ourselves now. > + */ This comment is hard to understand - what do you mean here? > + if (!rgi->worker_error) > + rgi->worker_error= 1; > + } I would have expected an abort (goto err) here (or well just below, after unlocking the mutex). Is this not needed, and if so, why not? > @@ -716,12 +724,20 @@ retry_event_group(rpl_group_info *rgi, rpl_parallel_thread *rpt, > unregistering (and later re-registering) the wait. > */ > if(thd->wait_for_commit_ptr) > - thd->wait_for_commit_ptr->unregister_wait_for_prior_commit(); > + thd->wait_for_commit_ptr->unregister_wait_for_prior_commit(); Indentation? - Kristian.

2 5

[Maria-developers] Developers Unconference in New York
by Ian Gilfillan 16 Feb '18

16 Feb '18

A final reminder about the Developers Unconference happening in New York on Sat 24 and Sun 25 February, just before M|18. https://mariadb.org/2018-mariadb-developers-unconference/ Please sign up on the Meetup page if you are going to be attending: https://www.meetup.com/MariaDB-Developers/events/244375856/ Hope to see you there!

1 0

[Maria-developers] Google Summer of Code 2018 - MariaDB Foundation accepted
by Colin Charles 14 Feb '18

14 Feb '18

Hi, I just noticed this piece of good news — MariaDB Foundation is accepted to participate in the Google Summer of Code 2018 again — https://summerofcode.withgoogle.com/ There is an already healthy list of ideas: https://mariadb.com/kb/en/library/google-summer-of-code-2018/ Many already have mentors from MariaDB Corporation or MariaDB Foundation. Some are looking for mentors. Happy coding this summer! cheers, -colin -- Colin Charles, http://bytebot.net/blog/ twitter: @bytebot | skype: colincharles "First they ignore you, then they laugh at you, then they fight you, then you win." -- Mohandas Gandhi

1 0

Re: [Maria-developers] [Commits] f4f32ab71d5: MDEV-9744: session optimizer_use_condition_selectivity=5 causing SQL Error (1918):
by Sergey Petrunia 13 Feb '18

13 Feb '18

Hi Varun, On Sat, Feb 03, 2018 at 06:37:21PM +0530, Varun wrote: > revision-id: f4f32ab71d5c96fbec87f8b6a5c057af2bc2398b (mariadb-10.2.5-413-gf4f32ab71d5) > parent(s): 9390ff53fc39e34976cf051ce33649f9316bb8e6 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-02-03 18:24:49 +0530 > message: > > MDEV-9744: session optimizer_use_condition_selectivity=5 causing SQL Error (1918): > Encountered illegal value '' when converting to DECIMAL > > The issue was that EITS data was allocated but then not read for some reason (one being to avoid a deadlock), > then the optimizer was using these bzero'ed buffers as EITS statistics. > This should not be allowed, we should use statistcs for a table only when we have successfully loaded/read > the stats from the statistical tables. > > --- > mysql-test/r/statistics.result | 16 ++++++++++++++++ > mysql-test/t/statistics.test | 19 +++++++++++++++++++ > sql/sql_statistics.cc | 23 ++++++++++++++++------- > 3 files changed, 51 insertions(+), 7 deletions(-) > > diff --git a/mysql-test/r/statistics.result b/mysql-test/r/statistics.result > index ffaaf09acc8..1486a8ab5e6 100644 > --- a/mysql-test/r/statistics.result > +++ b/mysql-test/r/statistics.result > @@ -1716,5 +1716,21 @@ b > 0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.004,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.004,0.000,0.000,0.004,0.000,0.000,0.000,0.004,0.000,0.000,0.000,0.004,0.000,0.000,0.000,0.004,0.000,0.004,0.000,0.000,0.004,0.000,0.004,0.000,0.004,0.000,0.004,0.004,0.004,0.000,0.004,0.000,0.004,0.004,0.004,0.004,0.004,0.000,0.004,0.004,0.004,0.004,0.004,0.004,0.008,0.004,0.008,0.008,0.008,0.008,0.020,0.004,0.016,0.020,0.016,0.016,0.051,0.031,0.027,0.031,0.043,0.047,0.043,0.043,0.055,0.051,0.071,0.043,0.043,0.043,0.020,0.024,0.024,0.020,0.016,0.016,0.008,0.008,0.012,0.000 > DROP TABLE t1; > # > +# MDEB-9744: session optimizer_use_condition_selectivity=5 causing SQL Error (1918): Please change to "MDEV" > +# Encountered illegal value '' when converting to DECIMAL > +# > +set @save_optimizer_use_condition_selectivity= @@optimizer_use_condition_selectivity; > +set optimizer_use_condition_selectivity=3, use_stat_tables=preferably; > +create table a (id int(10),cost decimal(9,2)) engine=innodb; > +ANALYZE TABLE a PERSISTENT FOR ALL; > +Table Op Msg_type Msg_text > +test.a analyze status Engine-independent statistics collected > +test.a analyze status OK > +create temporary table b (id int); > +insert into b (id) select id from a where cost > 0; > +drop table a,b; Please follow the convention and use table names t1,t2,... Ok to push after this is addressed. BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] MDEV-15167 Server crashes in in bitmap_bits_set upon REPAIR PARTITION ...
by Sergei Golubchik 10 Feb '18

10 Feb '18

Hi, Sachin! On Feb 09, Sachin Setiya wrote: > commit 3e49ebcacf9022931f0dfefcb44c13973128f8c4 > Author: Sachin Setiya <sachinsetia1001(a)gmail.com> > Date: Fri Feb 9 17:44:04 2018 +0530 > > MDEV-15167 Server crashes in in bitmap_bits_set upon REPAIR PARTITION > ... > > diff --git a/sql/sql_partition.cc b/sql/sql_partition.cc > index a4d9406..3d53056 100644 > --- a/sql/sql_partition.cc > +++ b/sql/sql_partition.cc > @@ -4583,6 +4583,11 @@ uint prep_alter_part_table(THD *thd, TABLE *table, > Alter_info *alter_info, > Alter_info::ALTER_TABLE_REORG | > Alter_info::ALTER_REBUILD_PARTITION)) > { > + /* > + You can't add column when we are doing alter related to partition > + */ > + DBUG_EXECUTE_IF("test_pseudo_invisible",{DBUG_ASSERT(0);}); > + DBUG_EXECUTE_IF("test_completely_invisible",{DBUG_ASSERT(0);}); May be return error? Like with P_S? DBUG_EXECUTE_IF("test_pseudo_invisible", {DBUG_RETURN(1)}); but test it, may be you'll need my_error too, like DBUG_EXECUTE_IF("test_pseudo_invisible", { my_error(ER_INTERNAL_ERROR, MYF(0), "Don't to it with test_pseudo_invisible"); DBUG_RETURN(1) }); Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] Oracle strings functions compatibility : substr / MDEV 10574
by jerome brauge 09 Feb '18

09 Feb '18

Hello Alexander, If you remember, there was still a problem with our function substr_oracle when length is less or equal to zero. Can you review this very small patch for this ? For MDEV-10574, If we overload only functions required for Oracle compatibility (ltrim/rtrim and lpad/rpad), could I make a similar patch ? (by adding functions ltrim_oracle, ... like for substr) It's for us the last blocking point. Regards, Jérôme. > -----Message d'origine----- > De : Alexander Barkov [mailto:bar@mariadb.org] > Envoyé : mardi 19 septembre 2017 13:16 > À : jerome brauge > Objet : Re: Oracle strings functions compatibility : substr > > Hi Jerome, > > I noticed one more difference in SUBSTR: > > > SELECT SUBSTR('aaa',1,-1) FROM DUAL; > > MariaDB returns an empty string. > Oracle returns NULL. > > Here's Oracle's documentation: > > https://docs.oracle.com/cd/B19306_01/server.102/b14200/functions162.htm > > > I think we should do this in the same patch, to avoid behavior change in the > future. > > Would you like to try doing this additional change? > Or would you like me to make an incremental patch on top of yours? > > Thanks! > > On 09/19/2017 10:33 AM, jerome brauge wrote: > > Hello Alexander, > > It's done. > > > > Do you have news from Sergei for MDEV-12874, MDEV-13417 and MDEV- > 13418 ? > > I asked Sergei, awaiting for his answer. I'll let you know. > > > > > Thank you very much. > > > > > > > >> -----Message d'origine----- > >> De : Alexander Barkov [mailto:bar@mariadb.org] Envoyé : lundi 18 > >> septembre 2017 18:12 À : jerome brauge Objet : Re: Oracle strings > >> functions compatibility : substr > >> > >> Hello Jerome, > >> > >> > >> On 09/18/2017 05:14 PM, jerome brauge wrote: > >>> Hello Alexander, > >>> I don't understand how I've missed such an obvious solution ! > >>> Here is the new patch. > >> > >> Thanks. Now it looks simpler. > >> > >> Can you please do one small thing: > >> move get_position() from "public:" to "protected:". > >> > >> Ok to make a pull request after this change. > >> > >> Thank you very much! > >> > >> > >>> > >>> Regards. > >>> Jérôme. > >>> > >>> > >>>> -----Message d'origine----- > >>>> De : Alexander Barkov [mailto:bar@mariadb.org] Envoyé : lundi 18 > >>>> septembre 2017 11:30 À : jerome brauge Objet : Re: Oracle strings > >>>> functions compatibility : substr > >>>> > >>>> Hello Jerome, > >>>> > >>>> > >>>> On 08/17/2017 04:43 PM, jerome brauge wrote: > >>>>> Hello Alexander, > >>>>> Here is a patch for function SUBSTR in sql_mode=oracle (If > >>>>> position is 0, > >>>> then it is treated as 1). > >>>> > >>>> I'm thinking of how to get a smaller patch. > >>>> > >>>> Wouldn't it be possible to introduce: > >>>> > >>>> virtual longlong get_position(); > >>>> > >>>> > >>>> > >>>> longlong Item_func_substr::get_position() { > >>>> return args[1]->val_int(); > >>>> } > >>>> > >>>> longlong Item_func_substr_oracle::get_position() > >>>> { > >>>> longlong pos= args[1]->val_int(); > >>>> return pos == 0 ? 1 : pos; > >>>> } > >>>> > >>>> > >>>> and just replace all calls for args[1]->val_int() to get_position() ? > >>>> > >>>> Thanks! > >>>> > >>>> > >>>>> > >>>>> Regards, > >>>>> Jérôme. > >>>>>

1 1

Re: [Maria-developers] 9bc03294c89: MDEV-15033: Node fails to join mdb cluster with wsrep_sst_mariabackup due to dependency from socat
by Sergei Golubchik 06 Feb '18

06 Feb '18

Hi, Jan! No, sorry, you cannot do it. First, we cannot add new RPM dependencies in GA (we tried once and had to revert that change). Second, there was already socat dependency in RPM, added by Nirbhay in MDEV-6211 MariaDB-Galera-server uses 'socat', but 'socat' is not in the dependency list commit bd5ca5e6093046d7e69cc59f0950a7af2c0a7bf2 removed by Nirbhay in MDEV-7233: Fix issue with missing dependency socat when installing MariaDB-galera-server on RedhatEL/OracleEL/(Others?) RPM based commit 41367e4023d0823624a461597813e6902437f096 I don't see a good solution for this issue. Adding socat to mariabackup package aren't much better, as mariabackup users don't really need socat either. May be just a better error message in the script? On Feb 06, jan wrote: > revision-id: 9bc03294c89b16df5f556b7d9fa4cb1eadae451a (mariadb-10.1.30-31-g9bc03294c89) > parent(s): 1cd1db9602476ec6864f4a521de97d788fe93fac > author: Jan Lindström > committer: Jan Lindström > timestamp: 2018-02-06 10:59:27 +0200 > message: > > MDEV-15033: Node fails to join mdb cluster with wsrep_sst_mariabackup due to dependency from socat > > Add socat dependency to cpack_rpm.cmake. Similar dependency already > is on debian/control. > > diff --git a/cmake/cpack_rpm.cmake b/cmake/cpack_rpm.cmake > index e80fb199f03..5b9c6726b5e 100644 > --- a/cmake/cpack_rpm.cmake > +++ b/cmake/cpack_rpm.cmake > @@ -170,7 +170,7 @@ SETA(CPACK_RPM_server_PACKAGE_REQUIRES > IF(WITH_WSREP) > SETA(CPACK_RPM_server_PACKAGE_REQUIRES > "galera" "rsync" "lsof" "grep" "gawk" "iproute" > - "coreutils" "findutils" "tar") > + "coreutils" "findutils" "tar" "socat") > IF (RPM MATCHES "sles11") > SETA(CPACK_RPM_server_PACKAGE_REQUIRES "util-linux") > ELSE() Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 2

[Maria-developers] Triangular FKs - Cascade delete leaves DB in state with broken referential integrity
by Oliver Schonrock 05 Feb '18

05 Feb '18

Initially posted on "discuss" list... I have come across a very strange symptom with triangular FKs (see below) where a cascade delete of the central (member) record leaves the DB in a state with broken referential integrity. The FK structure is a little bit demanding and somewhat "triangular/circular", but has been in operation in our application for many years without problems. After recent upgrade from 10.1 to 10.2.12, we promptly got broken FKs. (I have not yet retested with 10.1). How to reproduce: (example reduced to the bare essentials with 3 tables with 1 record each). SET FOREIGN_KEY_CHECKS=0; CREATE TABLE member ( id int(11) NOT NULL AUTO_INCREMENT, default_address_id int(11) DEFAULT NULL, default_card_id int(11) DEFAULT NULL, PRIMARY KEY (id), KEY member_FI_2 (default_address_id), KEY member_FI_3 (default_card_id), CONSTRAINT member_FK_2 FOREIGN KEY (default_address_id) REFERENCES address (id) ON DELETE SET NULL ON UPDATE CASCADE, CONSTRAINT member_FK_3 FOREIGN KEY (default_card_id) REFERENCES payment_method (id) ON DELETE SET NULL ON UPDATE CASCADE ) ENGINE=InnoDB; INSERT INTO member VALUES (1,2,3); CREATE TABLE address ( id int(11) NOT NULL AUTO_INCREMENT, member_id int(11) NOT NULL, PRIMARY KEY (id), KEY address_FI_1 (member_id), CONSTRAINT address_FK_1 FOREIGN KEY (member_id) REFERENCES member (id) ON DELETE CASCADE ON UPDATE CASCADE ) ENGINE=InnoDB; INSERT INTO address VALUES (2,1); CREATE TABLE payment_method ( id int(11) NOT NULL AUTO_INCREMENT, member_id int(11) NOT NULL, cardholder_address_id int(11) DEFAULT NULL, PRIMARY KEY (id), KEY payment_method_FI_1 (member_id), KEY payment_method_FI_2 (cardholder_address_id), CONSTRAINT payment_method_FK_1 FOREIGN KEY (member_id) REFERENCES member (id) ON DELETE CASCADE ON UPDATE CASCADE, CONSTRAINT payment_method_FK_2 FOREIGN KEY (cardholder_address_id) REFERENCES address (id) ON DELETE SET NULL ON UPDATE CASCADE ) ENGINE=InnoDB; INSERT INTO payment_method VALUES (3,1,2); SET FOREIGN_KEY_CHECKS=1; SELECT * FROM member; +----+--------------------+-----------------+ | id | default_address_id | default_card_id | +----+--------------------+-----------------+ | 1 | 2 | 3 | +----+--------------------+-----------------+ SELECT * FROM address; +----+-----------+ | id | member_id | +----+-----------+ | 2 | 1 | +----+-----------+ SELECT * FROM payment_method; +----+-----------+-----------------------+ | id | member_id | cardholder_address_id | +----+-----------+-----------------------+ | 3 | 1 | 2 | +----+-----------+-----------------------+ DELETE FROM member WHERE id = 1; SELECT * FROM member; [ empty ] SELECT * FROM address; [ empty ] SELECT * FROM payment_method; +----+-----------+-----------------------+ | id | member_id | cardholder_address_id | +----+-----------+-----------------------+ | 3 | 1 | NULL | +----+-----------+-----------------------+ ------------ DB is now in an inconsistent state: payment_method.member_id=1 references a non-existent member record. Am I missing something, or should this never happen? Thanks -- Oliver Schönrock

1 6

[Maria-developers] Partial indexes?
by Jared Beck 04 Feb '18

04 Feb '18

Dear Maria, :p Thanks for all your hard work! Any plans to implement partial indexes? (https://www.postgresql.org/docs/10/static/indexes-partial.html) I hope this is the right place to suggest new features. Please redirect me if not. -- Jared Beck • (607) 216-5373 • jared(a)jaredbeck.com

3 7

[Maria-developers] MDEV-14425: Is appending never slower than writing to a preallocated file?
by Marko Mäkelä 02 Feb '18

02 Feb '18

I would like to redesign the InnoDB redo log format for better performance in MariaDB 10.4. Part of this would involve minimizing write amplification and optimizing for journaled file systems. Jun Su from Microsoft suggested to me that regular writes (as opposed to appends) could cause some write amplification inside journaled file systems. InnoDB traditionally pre-allocates both data and log files. Maybe it was a good idea in 1994 when the code was initially conceived. But we have had journaled or copy-on-write file systems and also SSDs for quite some time now. I wrote two test programs that write a 2GiB file in 2KiB blocks, either pre-allocating the file upfront, or appending to the file. On the two SSDs that I tested (with ext4fs), appending was always faster. The programs are attached to https://jira.mariadb.org/browse/MDEV-14425 I would appreciate it if someone can provide a counterexample where writing to a preallocated file would be faster than appending, on a modern file system. Also, I would like to see how HDDs would perform. With best regards, Marko -- Marko Mäkelä, Lead Developer InnoDB MariaDB Corporation DON’T MISS M|18 MariaDB User Conference February 26 - 27, 2018 New York City https://m18.mariadb.com/

2 1

Re: [Maria-developers] [Commits] a0ea3a58224: MDEV-14857: problem with 10.2.11 server crashing when executing stored procedure
by Sergey Petrunia 31 Jan '18

31 Jan '18

Hi Sanja, Please find my review below. Lots of typos, but the two important comments are at the bottom, for the changes in sql_select.cc and sql_trigger.cc On Sat, Jan 27, 2018 at 11:39:10AM +0100, Oleksandr Byelkin wrote: > revision-id: a0ea3a5822486fa1702d6615f1be30684c3d412f (mariadb-10.1.30-26-ga0ea3a58224) > parent(s): 524221e7a34d42214e82dd868348b453f2ef1591 > author: Oleksandr Byelkin > committer: Oleksandr Byelkin > timestamp: 2018-01-27 11:38:38 +0100 > message: > > MDEV-14857: problem with 10.2.11 server crashing when executing stored procedure > > Counter for select numbering made stored with the statement (before was global) > So now it does have always accurate value which does not depend on > interruption of statement prepare by errors like lack of table in > a view definition. > > --- > diff --git a/sql/sql_class.h b/sql/sql_class.h > index 4556487bdfe..2dc2fb61e9d 100644 > --- a/sql/sql_class.h > +++ b/sql/sql_class.h > @@ -1027,6 +1027,21 @@ class Statement: public ilink, public Query_arena > > LEX_STRING name; /* name for named prepared statements */ > LEX *lex; // parse tree descriptor > + /* > + LEX whith represent curent statement (conventional, SP or PS) s/whith/which/ s/curent/current/ s/represent/represents/ > + > + For example during view parsing THD::lex will point to the views LEX and > + THD::stmt_lex will point on LEX of the statement where the view will be s/point on/point to/ > + included > + > + Now used to have alwais correct select numbering inside statemet s/Now used/Currently it is used/ s/alwais/always/ s/statemet/statement/ > + (LEX::current_select_number) without storing and restoring a global > + counter which was THD::select_number. > + > + TODO: make some unified statement representation (now SP has different) > + to store such data like LEX::current_select_number. > + */ > + LEX *stmt_lex; > /* > Points to the query associated with this statement. It's const, but > we need to declare it char * because all table handlers are written > diff --git a/sql/sql_lex.h b/sql/sql_lex.h > index b57fba08b47..a2da6f672ca 100644 > --- a/sql/sql_lex.h > +++ b/sql/sql_lex.h > @@ -727,7 +727,13 @@ class st_select_lex: public st_select_lex_node > Item *prep_having;/* saved HAVING clause for prepared statement processing */ > /* Saved values of the WHERE and HAVING clauses*/ > Item::cond_result cond_value, having_value; > - /* point on lex in which it was created, used in view subquery detection */ > + /* > + Point on lex in which it was created, used in view subquery detection. "Points to the LEX...". Also please shift the whole comment 1 position to the left. > + > + TODO: make also st_select_lex::parent_stmt_lex (see THD::stmt_lex) > + and use st_select_lex::parent_lex & st_select_lex::parent_stmt_lex > + instead of global (from THD) references where it is possible. > + */ > LEX *parent_lex; > enum olap_type olap; > /* FROM clause - points to the beginning of the TABLE_LIST::next_local list. */ > @@ -2452,7 +2458,7 @@ struct LEX: public Query_tables_list > /** SELECT of CREATE VIEW statement */ > LEX_STRING create_view_select; > > - uint number_of_selects; // valid only for view > + uint current_select_number; // valid for statmet LEX (not view) s/statmet/statement/ > > /** Start of 'ON table', in trigger statements. */ > const char* raw_trg_on_table_name_begin; > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index 6084c59a257..8d3116cd8b3 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -6903,7 +6903,12 @@ void THD::reset_for_next_command(bool do_clear_error) > clear_error(1); > > thd->free_list= 0; > - thd->select_number= 1; > + /* > + The same thd->stmt_lex made in lex_start, but during bootstrup this "We also assign thd->stmt_lex in lex_start()" s/bootstrup/bootstrap > + code executed first code is executed > + */ > + thd->stmt_lex= &main_lex; thd->stmt_lex->current_select_number= 1; > + DBUG_PRINT("info", ("Lex %p stmt_lex: %p", thd->lex, thd->stmt_lex)); > /* > Those two lines below are theoretically unneeded as > THD::cleanup_after_query() should take care of this already. > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index b9fe8f3162a..7a6a028ee9c 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -2462,6 +2462,17 @@ void JOIN::save_explain_data(Explain_query *output, bool can_overwrite, > bool need_tmp_table, bool need_order, > bool distinct) > { > + /* > + If there is SELECT in this statemet with the same number it must be the > + same SELECT > + */ > + DBUG_ASSERT(select_lex->select_number == UINT_MAX || > + select_lex->select_number == INT_MAX || > + !output || Is it really possible that output==NULL? If yes, that's probably a bug. > + !output->get_select(select_lex->select_number) || > + output->get_select(select_lex->select_number)->select_lex == > + select_lex); > + > if (select_lex->select_number != UINT_MAX && > select_lex->select_number != INT_MAX /* this is not a UNION's "fake select */ && > have_query_plan != JOIN::QEP_NOT_PRESENT_YET && > @@ -24601,6 +24612,11 @@ int JOIN::save_explain_data_intern(Explain_query *output, bool need_tmp_table, > { > explain= new (output->mem_root) Explain_select(output->mem_root, > thd->lex->analyze_stmt); > + if (!explain) > + DBUG_RETURN(1); // EoM > +#ifndef DBUG_OFF > + explain->select_lex= select_lex; > +#endif > join->select_lex->set_explain_type(true); > > explain->select_id= join->select_lex->select_number; > diff --git a/sql/sql_trigger.cc b/sql/sql_trigger.cc > index 0b4978b2862..878cd5a3b57 100644 > --- a/sql/sql_trigger.cc > +++ b/sql/sql_trigger.cc > @@ -1378,7 +1378,7 @@ bool Table_triggers_list::check_n_load(THD *thd, const char *db, > ulonglong save_sql_mode= thd->variables.sql_mode; > LEX_STRING *on_table_name; > > - thd->lex= &lex; > + thd->lex= thd->stmt_lex= &lex; > The same comment as before: we change thd->stmt_lex here. The function has code below that restores the value of thd->lex: // QQ: anything else ? lex_end(&lex); thd->lex= old_lex; But the value of thd->stmt_lex is not restored? > save_db.str= thd->db; > save_db.length= thd->db_length; BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] Wording suggestion?
by Sergei Golubchik 30 Jan '18

30 Jan '18

Hi, Russell! On Jan 30, Russell J.T. Dyer wrote: > How about concurrent_assign ? I think "concurrent" already has a clear meaning, at least in MariaDB - operations are done "concurrently" if they're executed truly in parallel, usually in different threads. For example, both MyISAM and InnoDB allow concurrent SELECTs, but only InnoDB allows concurrent UPDATEs. I considered simultaneous_assign, as a slightly shorter alternative to simultaneous_assignment. But it looked somewhat unnatural to me. Still, this is the case when I happily yield to the opinion of a documentation writer :) If you think that simultaneous_assign looks just fine, I'll use it. > > It's not only for update. There is a patch to fix SELECT ... INTO, > > and I suppose we'll have "simultaneous assignment" behavior > > everywhere eventually. > > > > So, I'm leaning towards either simultaneous_assignment or > > simultaneous_set as mode names. The first is more exact, but it's > > rather long (sql_mode names range from 3 to 26 characters, and this > > one is 23). The second (with 16) fits right in the middle of the > > range, length-wise. But there's no SET keyword in SELECT ... INTO. > > On the other hand, there's no ASSIGNMENT keyword anywhere either :) > > Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0