June 2020 - developers - lists.mariadb.org

[Maria-developers] Progress Report GSoC 2020 (15-20 June)
by Anton Mikhaylenko 24 Jun '20

24 Jun '20

Good day, everybody. In this post, I want to describe what I did this week, June 15-20. I`m working with threadpool concurrency. Have made some fixes. Change type of adjustment interval. Used the existing time interval to adjust the throttling. Regards, Anton MIkhailenko

1 0

[Maria-developers] Review for: MDEV-17399 Add support for JSON_TABLE, part #6
by Sergey Petrunia 22 Jun '20

22 Jun '20

Hi Alexey, > revision-id: 0d342ab173cdbf5d35c5f2833c66b6004b9e908e (mariadb-10.5.2-330-g0d342ab173c) > parent(s): b1ab211dee599eabd9a5b886fafa3adea29ae041 > author: Alexey Botchkov > committer: Alexey Botchkov > timestamp: 2020-06-08 14:51:04 +0400 > message: > > MDEV-17399 Add support for JSON_TABLE. > > ha_json_table handler implemented. > JSON_TABLE() added to SQL syntax. > First, some general input: == Assertion with Prepared Statements == prepare s from 'select * from json_table(?, \'$[*]\' columns( color varchar(100) path \'$.color\', price text path \'$.price\', seq for ordinality ) ) as T order by color desc; ' mysqld: /home/psergey/dev-git2/10.5-hf-review/sql/item.h:1002: virtual bool Item::fix_fields(THD*, Item**): Assertion `basic_const_item()' failed. == Default value for TEXT == Default value for VARCHAR(100) is SQL NULL, while for TEXT it is empty string. Is this intentional? If yes, I would like to see the spec of the logic behind this. select * from json_table('[{"color": "blue", "price": 50}, {"color": "red", "price": 100}]', '$[*]' columns(color varchar(100) path '$.nonexistent', seq for ordinality ) ) as T; +-------+------+ | color | seq | +-------+------+ | NULL | 1 | | NULL | 2 | +-------+------+ select * from json_table('[{"color": "blue", "price": 50}, {"color": "red", "price": 100}]', '$[*]' columns(color text path '$.nonexistent', seq for ordinality ) ) as T; +-------+------+ | color | seq | +-------+------+ | | 1 | | | 2 | +-------+------+ == JSON_TABLE can have a non-unique alias? == create table t20(a varchar(100)); insert into t20 values (); create table t21(a varchar(100)); insert into t21 values (); select * from t20 as T, t21 as T; ERROR 1066 (42000): Not unique table/alias: 'T' Good. This is the expected behaviour. select * from t20 as T , json_table(T.a, '$[*]' columns(color varchar(100) path '$.nonexistent', seq for ordinality ) ) as T; Empty set (0.000 sec) No error. I think this is wrong. Why would JSON_TABLE be allowed to use an alias that conflicts with another table? == EXPLAIN doesn't show table function == I had requested this in previous review(s): Table function use should be shown in EXPLAIN [FORMAT=JSON]. Here's how MySQL does it: https://gist.github.com/spetrunia/257c929b0c04a866faf6428e733999ad Note that this can be extended to handle other table functions. if there are no other ideas, let's use the same syntax. == Comments == I still think the patch doesn't have good comments. Let me get back to you on this. > diff --git a/sql/share/errmsg-utf8.txt b/sql/share/errmsg-utf8.txt > index 49cabec9916..3db438ee470 100644 > --- a/sql/share/errmsg-utf8.txt > +++ b/sql/share/errmsg-utf8.txt > @@ -7965,3 +7965,7 @@ ER_NOT_ALLOWED_IN_THIS_CONTEXT > eng "'%-.128s' is not allowed in this context" > ER_DATA_WAS_COMMITED_UNDER_ROLLBACK > eng "Engine %s does not support rollback. Changes where commited during rollback call" > +ER_JSON_TABLE_ERROR_ON_FIELD > + eng "Field '%s' can't be set for JSON_TABLE '%s'." > +ER_JSON_TABLE_DOUBLE_ON_RESPONSE > + eng "ON %s response specified more than once in JSON_TABLE definition." I agree that using this error solves the issue I've complained about, but this doesn't look like the right way to do it: * Why not just emit a parser error with YYERROR in this case? * Better: why not just remove the recursive definition from the grammar? ... > diff --git a/sql/table_function.cc b/sql/table_function.cc ... > + > +/* > + handler::print_error has a case statement for error numbers. > + This value is (10000) is far out of range and will envoke the > + default: case.(Current error range is 120-159 from include/my_base.h) > + */ > +#define HA_ERR_INVALID_JSON 10000 So, is this needed anymore? On the other hand, the new solution with returning HA_ERR_TABLE_IN_FK_CHECK doesn't look very good either. A plugin could register its own error codes, but this engine is not a plugin. Let's discuss with Serg and get back to this. > +static int print_path(String *str, const json_path_t *p) > +{ > + return str->append('\"') || > + str->append((const char *) p->s.c_str, p->s.str_end - p->s.c_str) || > + str->append('\"'); > +} The above doesn't do proper quoting, which is wrong. Testcase: select * from json_table('[{"co\\\\lor": "blue", "price": 50}]', '$[*]' columns( color varchar(100) path '$.co\\\\lor') ) as T; +-------+ | color | +-------+ | blue | +-------+ create view v2 as select * from json_table('[{"co\\\\lor": "blue", "price": 50}]', '$[*]' columns( color varchar(100) path '$.co\\\\lor') ) as T; select * from v2; +-------+ | color | +-------+ | NULL | +-------+ BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] Progress report for MDEV-12326
by Mohammed Hammaad Mateen 22 Jun '20

22 Jun '20

Greetings, Hope you are safe and doing great, This post describes the things I've done during 15-21 June under the mentor-ship of Sergei Golubchik and Oleksandr Byelkin for GSoC-20 to implement MDEV-12326. The tasks taken up for this week was to work towards extending the parser for multi-table DELETE....RETURNING and multi-table UPDATE....RETURNING, side-by-side, thereby changing the sql_yacc.yy file, for incorporating opt_returning rule in respective query format. I've also updated the test suite for multi-table UPDATE.... RETURNING and extended the parser by adding select_result parameter in multi-update and multi-delete functions, so as to enable returning a result set possible, whenever we query with RETURNING clause. All the code is uploaded up to date in these remote repositories, please feel free to comment. MULTI-TABLE UPDATE RETURNING: https://github.com/MariaDB/server/compare/10.5...MohammedHMateen:MDEV-12326… MULTI-TABLE DELETE RETURNING: https://github.com/MariaDB/server/compare/10.5...MohammedHMateen:MDEV-12326… Thank you Regards, Mohammed Hammaad Mateen

1 0

[Maria-developers] 97e932919bd: MDEV-22665: Print ranges in the optimizer trace created for non-indexed columns when optimizer_use_condition_selectivity >2
by Sergey Petrunia 17 Jun '20

17 Jun '20

Hi Varun, Please find some minor comments below. Ok to push when addressed. > revision-id: 97e932919bd53b854e0cea0d0001d185d7e63cc0 (mariadb-10.4.11-199-g97e932919bd) > parent(s): 0a5668f5128c731a346abf41afdc6fed33164ffc > author: Varun Gupta > committer: Varun Gupta > timestamp: 2020-05-26 22:36:03 +0530 > message: > > MDEV-22665: Print ranges in the optimizer trace created for non-indexed columns when optimizer_use_condition_selectivity >2 > > Now the optimizer trace shows the ranges constructed while getting estimates from EITS > > --- > mysql-test/main/opt_trace.result | 94 ++++++++++++++++++++++++++++++ > mysql-test/main/opt_trace.test | 18 ++++++ > sql/field.cc | 40 +++++++++++++ > sql/field.h | 1 + > sql/opt_range.cc | 123 ++++++++++++++++++++++++++++----------- > 5 files changed, 241 insertions(+), 35 deletions(-) ... > diff --git a/sql/opt_range.cc b/sql/opt_range.cc > index 66c870dd2ac..2cef483ef28 100644 > --- a/sql/opt_range.cc > +++ b/sql/opt_range.cc > @@ -15804,6 +15821,38 @@ void QUICK_GROUP_MIN_MAX_SELECT::dbug_dump(int indent, bool verbose) > > #endif /* !DBUG_OFF */ > > + > +/* > + @brief Print the comparison operator for the min range > +*/ > + > +static void print_min_range_operator(String *out, const key_range key) Do you really need to pass the whole struct key_range by value here? I think passing key.flag would be sufficient. > +{ > + if (key.flag == HA_READ_AFTER_KEY) > + out->append(STRING_WITH_LEN(" < ")); > + else if (key.flag == HA_READ_KEY_EXACT || > + key.flag == HA_READ_KEY_OR_NEXT) > + out->append(STRING_WITH_LEN(" <= ")); > + else > + out->append(STRING_WITH_LEN(" ? ")); > +} > + > + > +/* > + @brief Print the comparison operator for the max range > +*/ > + > +static void print_max_range_operator(String *out, const key_range key) > +{ Same as with print_min_range_operator: pass key.flag instead. > + if (key.flag == HA_READ_BEFORE_KEY) > + out->append(STRING_WITH_LEN(" < ")); > + else if (key.flag == HA_READ_AFTER_KEY) > + out->append(STRING_WITH_LEN(" <= ")); > + else > + out->append(STRING_WITH_LEN(" ? ")); > +} > + > + > static > void print_range(String *out, const KEY_PART_INFO *key_part, > KEY_MULTI_RANGE *range, uint n_key_parts) BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] Progress Report GSoC 2020 (8-13 June)
by Anton Mikhaylenko 17 Jun '20

17 Jun '20

Good day, everybody. In this post, I want to describe what I did this week, June 8-13. Added counting № of completed tasks for during period. During this week I tried to understand why threads dying fast. https://github.com/sapiest/server/commit/7ffa698044092e346353e4754d50188ef8… Regards, Anton MIkhailenko

1 0

Re: [Maria-developers] 3c81ed2f564: Add mytop to Client component for rpm package
by Sergei Golubchik 16 Jun '20

16 Jun '20

Hi, Anel! On Jun 16, Anel Husakovic wrote: > revision-id: 3c81ed2f564 (mariadb-10.5.2-359-g3c81ed2f564) > parent(s): a793ae5bc12 > author: Anel Husakovic <anel(a)mariadb.org> > committer: Anel Husakovic <anel(a)mariadb.org> > timestamp: 2020-06-06 08:21:01 +0200 > message: > > Add mytop to Client component for rpm package > > Part of MDEV-22552 This is ok. But don't push it yet, let's have all commits ok to push them all at once Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

1 0

Re: [Maria-developers] 4cbbc796dcc: MDEV-22420 DDL on temporary object is prohibited when XA is in prepare state
by Sergei Golubchik 16 Jun '20

16 Jun '20

Hi, Andrei! Ok to push! Thanks! On Jun 16, Andrei Elkin wrote: > >> > What about RENAME? What other operations are there that don't > >> > open a table? > >> > >> So RENAME is in above and all that fits to > >> stmt_causes_implicit_commit() that is flagged CF_AUTO_COMMIT_TRANS > >> (that involves CF_IMPLICIT_COMMIT_BEGIN). That reads every DDL, > >> Admin, Account that are bin-logged. > > > > I mean RENAME for temporary tables. Does it also cause an implicit > > commit? > > Yes, it does, as any syntax whose lex > lex->tmp_table() > returns false here. > > RENAME specifically does not have 'temporary' qualifier. Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

1 0

Re: [Maria-developers] 4cbbc796dcc: MDEV-22420 DDL on temporary object is prohibited when XA is in prepare state
by Sergei Golubchik 16 Jun '20

16 Jun '20

Hi, Andrei! On Jun 16, Andrei Elkin wrote: > > > What about RENAME? What other operations are there that don't open a table? > > So RENAME is in above and all that fits to stmt_causes_implicit_commit() > that is flagged CF_AUTO_COMMIT_TRANS (that involves > CF_IMPLICIT_COMMIT_BEGIN). > That reads every DDL, Admin, Account that are bin-logged. I mean RENAME for temporary tables. Does it also cause an implicit commit? Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

1 0

[Maria-developers] MariaDB support in WAL-G
by Кирилл Юлдашев 16 Jun '20

16 Jun '20

Hi! I implemented MariaDB support in the WAL-G backup system (Here is README https://github.com/wal-g/wal-g/blob/master/MySQL.md)) the implementation turned out to be compatible with MySQL. Can I ask someone from MariaDB to do a small design review and send it to me? I am defending my diploma on this topic on June 17 this year, and the developers ' opinion would be very useful. Integration tests: https://github.com/wal-g/wal-g/tree/master/docker/mariadb_tests/scripts/tes… Internal database stuff and command handlers: https://github.com/wal-g/wal-g/tree/master/internal/databases/mysql Commands: https://github.com/wal-g/wal-g/tree/master/cmd/mysql But PITR for MariaDB is not working yet, because of problems which occurred during the implementation stage due to the lack of documentation. I'll write about my problems later in another email. Thanks! Regards, Kirill

1 0

Re: [Maria-developers] 4cbbc796dcc: MDEV-22420 DDL on temporary object is prohibited when XA is in prepare state
by Sergei Golubchik 15 Jun '20

15 Jun '20

Hi, Andrei! On Jun 15, Andrei Elkin wrote: > revision-id: 4cbbc796dcc (mariadb-10.5.2-335-g4cbbc796dcc) > parent(s): a74e724e061 > author: Andrei Elkin <andrei.elkin(a)mariadb.com> > committer: Andrei Elkin <andrei.elkin(a)mariadb.com> > timestamp: 2020-06-11 17:24:15 +0300 > message: > > MDEV-22420 DDL on temporary object is prohibited when XA is in prepare state > > The parser must reject DDL operations on temporary objects when > they may modify or alter such object, including temporary tables and sequences. > The rejection is regardless (has been already in place for bin-loggable DML:s) > of the binlogging capability of the server or connection. > > The patch implements the requirement. A binlog test is added. > > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index 86f81dd001a..6ac0cf96024 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -4890,6 +4890,11 @@ mysql_execute_command(THD *thd) > } > else > { > + if (thd->transaction->xid_state.check_has_uncommitted_xa()) > + { > + thd->transaction->xid_state.er_xaer_rmfail(); > + goto error; > + } > status_var_decrement(thd->status_var.com_stat[lex->sql_command]); > status_var_increment(thd->status_var.com_drop_tmp_table); Two comments. 1. Where is this checked for non-temporary tables? check_has_uncommitted_xa is tested in open_tables(), and it's not called for DROP. What about RENAME? What other operations are there that don't open a table? 2. check_has_uncommitted_xa() already does er_xaer_rmfail() internally, you don't need to repeat it. Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

1 0