June 2020 - developers - lists.mariadb.org

Re: [Maria-developers] 5ff9525ff86: MDEV-22722 Assertion "inited==NONE" failed in handler::ha_index_init on the slave during UPDATE
by Sergei Golubchik 10 Jun '20

10 Jun '20

Hi, Sachin! On Jun 10, Sachin wrote: > revision-id: 5ff9525ff86 (mariadb-10.5.2-290-g5ff9525ff86) > parent(s): ff881056b8d > author: Sachin <sachin.setiya(a)mariadb.com> > committer: Sachin <sachin.setiya(a)mariadb.com> > timestamp: 2020-06-07 16:45:01 +0530 > message: > > diff --git a/sql/log_event_server.cc b/sql/log_event_server.cc > index 5910ece12f0..ffda352c866 100644 > --- a/sql/log_event_server.cc > +++ b/sql/log_event_server.cc > @@ -5570,6 +5570,12 @@ int Rows_log_event::do_apply_event(rpl_group_info *rgi) > { > master_had_triggers= table->master_had_triggers; > bool transactional_table= table->file->has_transactions_and_rollback(); > + if (get_general_type_code() == WRITE_ROWS_EVENT) > + table->file->prepare_for_insert(0); > + else if (get_general_type_code() == UPDATE_ROWS_EVENT || > + get_general_type_code() == DELETE_ROWS_EVENT) > + table->file->prepare_for_insert(1); > + You can simplify it as table->file->prepare_for_insert(get_general_type_code() != WRITE_ROWS_EVENT); otherwise ok to push Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

1 0

Re: [Maria-developers] MDEV-21831: Assertion `length == pack_length()' failed in Field_inet6::sort_string upon INSERT into RocksDB table
by Sergey Petrunia 09 Jun '20

09 Jun '20

Hi Varun, > MDEV-21831: Assertion `length == pack_length()' failed in Field_inet6::sort_string upon INSERT into RocksDB table > > For INET6 columns the values are stored as BINARY columns and returned to the client in TEXT format. > For rocksdb the indexes store mem-comparable images for columns, so use the pack_length() to store > the mem-comparable form for INET6 columns. This would also remain consistent with CHAR columns. > > diff --git a/storage/rocksdb/rdb_datadic.cc b/storage/rocksdb/rdb_datadic.cc > index 21a9b57b555e..6e73535d1de7 100644 > --- a/storage/rocksdb/rdb_datadic.cc > +++ b/storage/rocksdb/rdb_datadic.cc > @@ -3317,7 +3317,9 @@ bool Rdb_field_packing::setup(const Rdb_key_def *const key_descr, > field->field_length = field->char_length() * cs->mbmaxlen. > */ > const CHARSET_INFO *cs = field->charset(); > - m_max_image_len = cs->strnxfrmlen(field->field_length); > + m_max_image_len = cs->strnxfrmlen(type == MYSQL_TYPE_STRING ? > + field->pack_length() : > + field->field_length); Field::pack_length is defined as: /* pack_length() returns size (in bytes) used to store field data in memory (i.e. it returns the maximum size of the field in a row of the table, which is located in RAM). */ virtual uint32 pack_length() const { return (uint32) field_length; } so, it is size in bytes, while strnxfrmlen() accepts length *in characters*. That is, strnxfrmlen(field->pack_length()) is certainly not correct. Rdb_field_packing::m_max_image_len is defined as: /* Length of mem-comparable image of the field, in bytes */ int m_max_image_len; while pack_length() seems to describe table->record[0] format (or is it KeyTupleFormat) ? Field::sort_length() doesn't have a definition but it seems to be a closer match as it describes the length of the mem-comparable form? BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] 06942363ebc: MDEV-14347 CREATE PROCEDURE returns no error when using an unknown variable
by Sergei Golubchik 09 Jun '20

09 Jun '20

Hi, Alexander! On Jun 07, Alexander Barkov wrote: > revision-id: 06942363ebc (mariadb-10.5.2-311-g06942363ebc) > parent(s): 0d6d63e1505 > author: Alexander Barkov <bar(a)mariadb.com> > committer: Alexander Barkov <bar(a)mariadb.com> > timestamp: 2020-06-03 12:11:07 +0400 > message: > > MDEV-14347 CREATE PROCEDURE returns no error when using an unknown variable > > diff --git a/sql/item.cc b/sql/item.cc > index 8ea6366e6c4..5800b9bc95e 100644 > --- a/sql/item.cc > +++ b/sql/item.cc > @@ -764,6 +764,29 @@ bool Item_field::collect_item_field_processor(void *arg) > } > > > +void Item_ident::undeclared_spvar_error() const Currently it's used only in Item_field::unknown_splocal_processor(). Do you plan to use this somewhere else? Why is it a separate method and in Item_ident? > +{ > + /* > + We assume this is an unknown SP variable, possibly a ROW variable. > + Print the leftmost name in the error: > + SET var=a; -> a > + SET var=a.b; -> a > + SET var=a.b.c; -> a > + */ > + my_error(ER_SP_UNDECLARED_VAR, MYF(0), db_name.str ? db_name.str : > + table_name.str ? table_name.str : > + field_name.str); > +} > + > +bool Item_field::unknown_splocal_processor(void *arg) > +{ > + DBUG_ENTER("Item_field::unknown_splocal_processor"); > + DBUG_ASSERT(type() == FIELD_ITEM); > + undeclared_spvar_error(); > + DBUG_RETURN(true); > +} > + > + > bool Item_field::add_field_to_set_processor(void *arg) > { > DBUG_ENTER("Item_field::add_field_to_set_processor"); > @@ -8012,6 +8035,27 @@ void Item_ref::cleanup() > } > > > +bool Item_ref::unknown_splocal_processor(void *arg) > +{ > + DBUG_ENTER("Item_ref::unknown_splocal_processor"); > + DBUG_ASSERT(type() == REF_ITEM); > + DBUG_ASSERT(ref_type() == REF); > + undeclared_spvar_error(); > + DBUG_RETURN(true); > +} > + > + > +bool Item_ref::walk(Item_processor processor, bool walk_subquery, void *arg) > +{ > + if (processor == &Item::unknown_splocal_processor) > + return unknown_splocal_processor(arg); Why is that? This is very unusual, normally walk() should not change its behavior based on the processor. > + if (ref && *ref) > + return (*ref)->walk(processor, walk_subquery, arg) || > + (this->*processor)(arg); > + return false; > +} > + > + > /** > Transform an Item_ref object with a transformer callback function. > Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

2 2

Re: [Maria-developers] JSON aggregate functions working on JSON documents
by Sergey Petrunia 08 Jun '20

08 Jun '20

On Mon, Jun 08, 2020 at 04:05:48PM +0400, Alexey Botchkov wrote: > So the problem is that we (MariaDB) doesn't consider the JSON field content > as a JSON, but as a plain text. > That affects not only aggregate functions, but any other expecting JSON > arguments. > To me it's the bug, should be fixed. Will you report it, Sergei? Filed: https://jira.mariadb.org/browse/MDEV-22837 BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] Progress Report GSoC 2020 (1-6 June)
by Anton Mikhaylenko 08 Jun '20

08 Jun '20

Good day, everybody. In this post, I want to describe what I did this week, June 1-6. I have added the initial implementation of the hill_climbing class. It includes an update function - the main function that changes the required number of threads. To implement this function, I added the Complex class - the base class for working with complex numbers. https://github.com/sapiest/server/commits/10.5-update_threadpool_concurrency The main work is not completed yet, as the algorithm requires adjustments and adaptation in MariaDB environment. Regards, Anton MIkhailenko

1 0

[Maria-developers] JSON aggregate functions working on JSON documents
by Sergey Petrunia 08 Jun '20

08 Jun '20

Hi Alexey, I'm looking at how JSON_ARRAYAGG and JSON_OBJECTAGG work at processing JSON documents. I'm trying these examples, first on MySQL-8: create table t30 (col1 json); insert into t30 values('{"color":"red", "size":1}' ); insert into t30 values('{"color":"blue", "size":2}' ); select JSON_ARRAYAGG(col1) from t30; +-------------------------------------------------------------+ | JSON_ARRAYAGG(col1) | +-------------------------------------------------------------+ | [{"size": 1, "color": "red"}, {"size": 2, "color": "blue"}] | +-------------------------------------------------------------+ Good. create table t31 (name varchar(100), contents json); insert into t31 values('obj1', '{"color":"blue", "size":2}' ); insert into t31 values('obj2', '{"color":"red", "size":1}' ); select JSON_OBJECTAGG(name, contents) from t31; +-----------------------------------------------------------------------------+ | JSON_OBJECTAGG(name, contents) | +-----------------------------------------------------------------------------+ | {"obj1": {"size": 2, "color": "blue"}, "obj2": {"size": 1, "color": "red"}} | +-----------------------------------------------------------------------------+ Good. Now, trying these on MariaDB, I get: MariaDB [test]> select JSON_ARRAYAGG(col1) from t30; +------------------------------------------------------------------------+ | JSON_ARRAYAGG(col1) | +------------------------------------------------------------------------+ | ["{\"color\":\"red\", \"size\":1}","{\"color\":\"blue\", \"size\":2}"] | +------------------------------------------------------------------------+ MariaDB [test]> select JSON_OBJECTAGG(name, contents) from t31; +---------------------------------------------------------------------------------------+ | JSON_OBJECTAGG(name, contents) | +---------------------------------------------------------------------------------------+ | {"obj1":"{\"color\":\"blue\", \"size\":2}", "obj2":"{\"color\":\"red\", \"size\":1}"} | +---------------------------------------------------------------------------------------+ The JSON documents are coverted into strings. I think, the users will prefer MySQL's behavior. But which behaivor is "correct"? (i.e. required by the standard, etc) Is MariaDBs's behaivor a bug? Is this expected? BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] MDEV-11563: GROUP_CONCAT(DISTINCT ...) may produce a non-distinct list
by Sergey Petrunia 07 Jun '20

07 Jun '20

Hi Varun, > revision-id: c408c41230c4a9de0114d13aaf861d3ced65f4c7 (mariadb-10.5.0-424-gc408c41230c) > parent(s): 95da2113a050ad739fdaf60ee871329468a01554 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2020-03-31 11:57:37 +0530 > message: > > MDEV-11563: GROUP_CONCAT(DISTINCT ...) may produce a non-distinct list > The commit message should describe the error and the fix made. (especially important here as this patch fixes only a part of the issue with GROUP_CONCAT) > --- > mysql-test/main/func_gconcat.result | 53 ++++++++++++++++++++++--------------- > mysql-test/main/func_gconcat.test | 7 +++++ > sql/item_sum.cc | 40 +++++++++++++++++----------- > sql/item_sum.h | 3 ++- > 4 files changed, 65 insertions(+), 38 deletions(-) > > diff --git a/sql/item_sum.h b/sql/item_sum.h > index e221d04397d..b10d9f5974d 100644 > --- a/sql/item_sum.h > +++ b/sql/item_sum.h > @@ -1879,7 +1879,8 @@ class Item_func_group_concat : public Item_sum > bool warning_for_row; > bool always_null; > bool force_copy_fields; > - bool no_appended; > + /** True if result has been written to output buffer. */ > + bool m_result_finalized; Why does this variable use the m_ convention when others do not? I think this is confusing (I stop for a moment to wonder how is it different from all others every time I see the name). Please remove this. What's "output buffer"? this->result ? And what does "result" mean here - the entire return value of GROUP_CONCAT function or some part of it? Please improve the comment. > /** Limits the rows in the result */ > Item *row_limit; > /** Skips a particular number of rows in from the result*/ Ok to push after this is addressed. BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] Is it okay that JSON_OBJECTAGG creates objects with duplicate elements?
by Sergey Petrunia 07 Jun '20

07 Jun '20

Hi Alexey, Consider this example: create table t20 (a varchar(10), b int); insert into t20 values ('red', 1), ('blue', 2), ('red',1001), ('blue', 1002); select json_objectagg(a,b) from t20; +----------------------------------------------+ | json_objectagg(a,b) | +----------------------------------------------+ | {"red":1, "blue":2, "red":1001, "blue":1002} | +----------------------------------------------+ the created object has multiple members with the same name. Is this expected or should be reported as a bug? In MySQL-8, the above example produces this: +-----------------------------+ | json_objectagg(a,b) | +-----------------------------+ | {"red": 1001, "blue": 1002} | +-----------------------------+ BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] 0f91c62d332: MDEV-22722 Assertion "inited==NONE" failed in handler::ha_index_init on the slave during UPDATE
by Sergei Golubchik 07 Jun '20

07 Jun '20

Hi, Sachin! Two questions: 1. This looks like there was no replication tests for periods of long uniques at all? Please add them (create an MDEV, like "replication tests of long unique") and add various relevant replication tests in 10.4. 2. It feels a bit risky to backport my changes to 10.4. At least you definitely should take the latest code from 10.5. Not cherry-pick one commit, but copy handler::prepare_for_insert() and other relevant code from 10.5. May be it'd be safer not to backport? What would your fix look like if you wouldn't have prepare_for_insert()? Also, a couple of comments to the code, see below. On Jun 03, Sachin wrote: > revision-id: 0f91c62d332 (mariadb-10.4.11-225-g0f91c62d332) > parent(s): c01b3d91690 > author: Sachin <sachin.setiya(a)mariadb.com> > committer: Sachin <sachin.setiya(a)mariadb.com> > timestamp: 2020-06-02 08:24:24 +0530 > message: > > MDEV-22722 Assertion "inited==NONE" failed in handler::ha_index_init on the slave during UPDATE > > Add missing call for handler->prepare_for_insert() in rows event. > > diff --git a/mysql-test/main/long_unique_bugs_replication.test b/mysql-test/main/long_unique_bugs_replication.test > new file mode 100644 > index 00000000000..5348be3e7d3 > --- /dev/null > +++ b/mysql-test/main/long_unique_bugs_replication.test > @@ -0,0 +1,22 @@ > +# > +# Long unique bugs related to master slave replication > +# > + > +# > +# MDEV-22722 Assertion "inited==NONE" failed in handler::ha_index_init on the slave during UPDATE > +# > + > +--source include/master-slave.inc > +--source include/have_binlog_format_row.inc master-slave.inc should always be included last, after all have_xxx.inc files that may cause the test to be skipped. > + > +create table t1 (i1 int, a1 text, unique key i1 (a1)) engine=myisam; > +insert into t1 values (1,1); > +update t1 set a1 = 'd' limit 1; > + > +sync_slave_with_master; > +connection slave; > + > +connection master; > +drop table t1; > + > +--source include/rpl_end.inc > diff --git a/sql/log_event.cc b/sql/log_event.cc > index 3707f73a716..f4c0c546a38 100644 > --- a/sql/log_event.cc > +++ b/sql/log_event.cc > @@ -11568,6 +11568,7 @@ int Rows_log_event::do_apply_event(rpl_group_info *rgi) > { > master_had_triggers= table->master_had_triggers; > bool transactional_table= table->file->has_transactions(); > + table->file->prepare_for_insert(); > /* > table == NULL means that this table should not be replicated > (this was set up by Table_map_log_event::do_apply_event() > @@ -14344,6 +14345,7 @@ int Rows_log_event::find_row(rpl_group_info *rgi) > issue_long_find_row_warning(get_general_type_code(), m_table->alias.c_ptr(), > is_index_scan, rgi); > table->default_column_bitmaps(); > + table->file->prepare_for_insert(); Why do you need it here, I'd think that do_apply_event() should be enough? > DBUG_RETURN(error); > } Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

2 1

Re: [Maria-developers] MDEV-16230: Server crashes when Analyze format=json is run with a window function with empty PARTITION BY and ORDER BY clauses
by Sergey Petrunia 03 Jun '20

03 Jun '20

Hi Varun, * Please limit the commit comment to ~80 chars wide. * The .result file has r_total_time_ms. Use analyze-format.inc to avoid this. Ok to push after this is addressed. > commit 9e755e21799695b9a837a240a86ba5ef863ae7b3 > Author: Varun Gupta <varun.gupta(a)mariadb.com> > Date: Tue Jun 2 16:31:53 2020 +0530 > > MDEV-16230: Server crashes when Analyze format=json is run with a window function with empty PARTITION BY and ORDER BY clauses > > Currently when both PARTITION BY and ORDER BY clauses are empty then we create a Item with the first field > in the select list and sort with that field. It should be created as an Item_temptable_field instead of > Item_field because the print() function continues to work even if the table has been dropped. > ... BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0