- developers - lists.mariadb.org

Re: [Maria-developers] b97e45651d1: MDEV-16937 Strict SQL with system versioned tables causes issues
by Sergei Golubchik 19 May '20

19 May '20

Hi, Aleksey! First - is commit b97e45651d1 the one I was supposed to review? On May 17, Aleksey Midenkov wrote: > revision-id: b97e45651d1 (mariadb-10.4.7-33-gb97e45651d1) > parent(s): 7587975bf06 > author: Aleksey Midenkov <midenok(a)gmail.com> > committer: Aleksey Midenkov <midenok(a)gmail.com> > timestamp: 2019-08-19 11:58:07 +0300 > message: > > MDEV-16937 Strict SQL with system versioned tables causes issues > > Vers SQL: respect system fields in NO_ZERO_DATE mode. > > This is the subject for refactoring in tempesta-tech/mariadb#379 > > diff --git a/mysql-test/suite/versioning/engines.combinations b/mysql-test/suite/versioning/engines.combinations > index 26b5bab23f1..57e2af6cd06 100644 > --- a/mysql-test/suite/versioning/engines.combinations > +++ b/mysql-test/suite/versioning/engines.combinations > @@ -7,5 +7,10 @@ default-storage-engine=innodb > [myisam] > default-storage-engine=myisam > > +[traditional] > +default-storage-engine=myisam > +sql-mode=traditional I'm not sure about it. It's, of course, safe to run all tests with every possible settings, but it really blows up testing times. May be it'd make sense to have a more targeted test here? > + > [heap] > default-storage-engine=memory > + > diff --git a/mysql-test/suite/versioning/r/select.result b/mysql-test/suite/versioning/r/select.result > index 3569268ce1d..68df246af6b 100644 > --- a/mysql-test/suite/versioning/r/select.result > +++ b/mysql-test/suite/versioning/r/select.result > @@ -45,7 +45,7 @@ ASOF_x y > 7 107 > 8 108 > 9 109 > -select x as FROMTO_x, y from t1 for system_time from timestamp '0-0-0 0:0:0' to timestamp @t1; > +select x as FROMTO_x, y from t1 for system_time from timestamp '1970-01-01 00:00:00' to timestamp @t1; Does this test use a fixed time zone? If not, is 1970-01-01 00:00:00 a valid timestamp in all time zones? If unsure, better use 1971, that will definitely be fine everywhere. > FROMTO_x y > 0 100 > 1 101 > diff --git a/sql/field.cc b/sql/field.cc > index 969c32a5180..f5580239a9f 100644 > --- a/sql/field.cc > +++ b/sql/field.cc > @@ -11139,6 +11139,13 @@ bool Field::save_in_field_default_value(bool view_error_processing) > { > THD *thd= table->in_use; > > + /* > + TODO: refactor setting the system fields via default_value mechanism. > + This condition will go away as well as other conditions with VERS_SYSTEM_FIELD. > + */ MDEV? There's a pull request, but it is marked as obsolete > + if (flags & VERS_SYSTEM_FIELD) > + return false; > + > if (unlikely(flags & NO_DEFAULT_VALUE_FLAG && > real_type() != MYSQL_TYPE_ENUM)) > { > diff --git a/sql/unireg.cc b/sql/unireg.cc > index d019b5f8a75..484ad0334e4 100644 > --- a/sql/unireg.cc > +++ b/sql/unireg.cc > @@ -1060,7 +1060,8 @@ static bool make_empty_rec(THD *thd, uchar *buff, uint table_options, > !f_bit_as_char(field->pack_flag)) > null_count+= field->length & 7; > > - error= make_empty_rec_store_default(thd, regfield, field->default_value); > + if (!(field->flags & VERS_SYSTEM_FIELD)) > + error= make_empty_rec_store_default(thd, regfield, field->default_value); will the field be initialized at all? It should have something, even bzero will do, but we shouldn't dump uninited memory to frm. > delete regfield; // Avoid memory leaks > if (error) > goto err; > Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

2 3

[Maria-developers] Progress report - Week 2
by Mohammed Hammaad Mateen 18 May '20

18 May '20

Greetings, Hope you are safe and doing great, This post describes the things I've done in my second week [ 11-17 May ] of Community Bonding Period under the mentor-ship of Sergei Golubchik and Oleksandr Byelkin for GSoC-20. Tasks done this week were mostly the continuation of the leftover tasks from the previous week and some related study. Things I've done include: 1:* I was caught up in analysing how was DELETE .... RETURNING implemented, which led me to read from:* • *sp_head.cc* : this made me understand that sp actually meant stored procedures but there is another called PS which I'm yet to figure out. • *sql_parse.cc* : from here I understood that delete return as actually { Analyse->Delete->Return } and all the logic for returning is executed if(lex->has_returning()). 2: Though I've raised the pull request, *MDEV-14558* is not yet completely tested as it does not include tests for *mysql \s*, moreover, I'm facing problems to escape the backslash inorder to write a test-case for *mysql \s* . Regards, Mohammed Hammaad Mateen

1 0

Re: [Maria-developers] 82b6a5cb85f: MDEV-22556: Incorrect result for window function when using encrypt-tmp-files=ON
by Sergei Golubchik 17 May '20

17 May '20

Hi, Varun! Ok to push. One comment below On May 17, Varun Gupta wrote: > revision-id: 82b6a5cb85f (mariadb-10.4.11-192-g82b6a5cb85f) > parent(s): 3bfe305c5cd > author: Varun Gupta <varun.gupta(a)mariadb.com> > committer: Varun Gupta <varun.gupta(a)mariadb.com> > timestamp: 2020-05-15 02:37:16 +0530 > message: > > MDEV-22556: Incorrect result for window function when using encrypt-tmp-files=ON > > diff --git a/mysql-test/suite/encryption/t/tempfiles_encrypted.opt b/mysql-test/suite/encryption/t/tempfiles_encrypted.opt > new file mode 100644 > index 00000000000..f515f7b8689 > --- /dev/null > +++ b/mysql-test/suite/encryption/t/tempfiles_encrypted.opt > @@ -0,0 +1,5 @@ > +--encrypt-tmp_files=ON > +--loose-file-key-management-encryption-algorithm=aes_cbc > +--file-key-management=FORCE > +--loose-file-key-management-filename=$MTR_SUITE_DIR/t/filekeys-data.enc > +--loose-file-key-management-filekey=FILE:$MTR_SUITE_DIR/t/filekeys-data.key this is very redundant, you include have_file_key_management_plugin.inc and it already sets all options for the file_key_management plugin to work. so you only need --encrypt-tmp_files=ON here. Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

1 0

[Maria-developers] Progress Report - Week 1-2
by Anton Mikhaylenko 17 May '20

17 May '20

Good day, everyone. In this post, I want to describe the things I've done in the 1-2 weeks of GSoC-20. For begin, I messaged with my mentor Vladislav Vaintroub and I got some tips on where to start. The first step was to run the server and tests for MariaDB on Linux and Windows. At this stage, a little bug was found, which was documented by my mentor https://jira.mariadb.org/browse/MDEV-22479. The next step was to look at the theoretical part of my work. Specifically, what algorithm to use for dynamic concurrency. This is the algorithm "hill-climbing", which is used in .NET ThreadPool https://www.researchgate.net/publication/228977836_Optimizing_concurrency_l… . In the last step, I started to get into the work of the current ThreadPool in MariaDB. At the moment this step is not completed yet. What needs to be done next: Fully understand the theoretic part and how the current thread pool works. Regards, Anton MIkhailenko

1 0

Re: [Maria-developers] Review for MDEV-19751.
by Aleksey Midenkov 14 May '20

14 May '20

Hi, Alexey! Actually the bug is applicable only to primary key and only when default field list is set (i.e. empty KEY() clause). So updated the patch accordingly. Please peek again. On Wed, May 13, 2020 at 2:40 PM Alexey Botchkov <holyfoot(a)mariadb.com> wrote: > > Hello, Aleksey! > > > commit 5ef168e00f7b8c821463e46535a4beccb47ec8ea > + if (key_info->flags & HA_INVISIBLE_KEY) > + continue; > > I'd add || !(key_info->flags & HA_NOSAME). > Can the non-unique index participate in the KEY partitioning? > > + for (uint kp= 0; kp < key_info->user_defined_key_parts; ++kp) > + { > + const KEY_PART_INFO &key_part= key_info->key_part[kp]; > + for (Field **part_field= tab_part_info->part_field_array; > + *part_field; ++part_field) > + { > + if (*part_field == key_part.field) > + { > + *partition_changed= TRUE; > + goto search_finished; > + } > + } // for (part_field) > + } // for (key_part) > > So you check if the key has any of the partitioning fields in it. > In this case we are going to disallow INPLACE with no reason if the > unrelated key is removed just containing the primary key column. > I'd say we should check for the exact match, when the key has > all the part_field_array in that exact order. Disagree? > > And for the test - please add the case for a table with no PRIMARY KEY, bug with the UNIQUE instead. > > Best regards! > HF > -- All the best, Aleksey Midenkov @midenok

1 0

[Maria-developers] Progress Report - Week 1
by Mohammed Hammaad Mateen 12 May '20

12 May '20

Greetings, Hope you are safe and doing great, This post describes the things I've done in my first week [ 4-10 May ] of Community Bonding Period under the mentor-ship of Sergei Golubchik and Oleksandr Byelkin for GSoC-20 and I must acknowledge that it was really an interactive experience and at the same time informative as well. Things I've done include: 1: *Raising my first ever pull request for MariaDB under MDEV-14558 *: Learnt about *strnxmov*, a function of C++ which I had never used earlier. I also came across the procedure of testing my implemented code. Wrote some test cases for the same and generated their corresponding results. 2:* Understand the working of the functions under multi_delete*: Ran it in the debugger and tried to familiarize myself with the internals of how Multiple Table Delete actually works. Things to-do: 1: Though I've raised the pull request, MDEV-14558 is not completed tested as it does not include tests for *mysql \s*, moreover, test cases have to be refactored. 2: Learn more about the working of *multi_delete* so as to develop a parser for Multiple Table Delete Returning. 3: Write some test cases for Multiple Table Delete Returning before hand. Regards, Mohammed Hammaad Mateen

2 1

[Maria-developers] Review for: MDEV-17399 Add support for JSON_TABLE, part #3
by Sergey Petrunia 10 May '20

10 May '20

Hi Alexey, Please find the next batch of input below. (I haven't finished the review but I'm sending it now so you can start addressing it) > commit 68ca18518337443090bdeddf6d612129b6843c21 > Author: Alexey Botchkov <holyfoot(a)askmonty.org> > Date: Fri Apr 17 14:42:40 2020 +0400 > > MDEV-17399 Add support for JSON_TABLE. > > Syntax for JSON_TABLE added. > The ha_json_table handler added. Executing the JSON_TABLE we > create the temporary table of the ha_json_table, add dependencies > of other tables and sometimes conditions to WHERE. > ... > +public: > + ha_json_table(TABLE_SHARE *share_arg, Table_function_json_table *jt): > + handler(&table_function_hton.m_hton, share_arg), m_jt(jt) > + { > + mark_trx_read_write_done= 1; This definitely needs a comment. > + ~ha_json_table() {} > + handler *clone(const char *name, MEM_ROOT *mem_root) { return NULL; } > + const char *index_type(uint inx) { return "HASH"; } Why is ha_json_table still trying to pretend it supports indexes? Is this a left-over from the old way of interfacing with the optimizer that can now be removed? > +int Table_function_json_table::setup(THD *thd, TABLE_LIST *sql_table, > + COND **cond) > +{ > + if (m_json->fix_fields(thd, &m_json)) > + return TRUE; > + > + m_dep_tables= m_json->used_tables(); Why does this function have a 'cond' argument? Please remove it if it's not necessary. == Duplicate column names are allowed == select * from JSON_TABLE('{"color": "black", "price": 1000}', "$" COLUMNS (rowSeq FOR ORDINALITY, color VARCHAR(100) PATH '$.color', color VARCHAR(100) PATH '$.price') ) as T; +--------+-------+-------+ | rowSeq | color | color | +--------+-------+-------+ | 0 | black | 1000 | +--------+-------+-------+ Is it really okay that duplicate column names are allowed? == RIGHT JOIN is allowed == create table t1 (item_name varchar(32), item_props varchar(1024)); insert into t1 values ('Laptop', '{"color": "black", "price": 1000}'); insert into t1 values ('Jeans', '{"color": "blue", "price": 50}'); This query should not be allowed, because JSON_TABLE has a dependency on an inner side of an outer join: select * from t1 right join json_table(t1.item_props, '$' columns(color varchar(100) path '$.color', f2 for ordinality) ) as T on 1; === NATURAL JOIN crashes == create table t10 (color varchar(100), f2 int); select * from t10 natural join json_table(t1.item_props, '$' columns(color varchar(100) path '$.color', f2 for ordinality) ) as JT ; Thread 31 "mysqld" received signal SIGSEGV, Segmentation fault. 0x0000555555d8ac12 in Select_limit_counters::get_select_limit (this=0xa5a5a5a5a5a5ad2d) at /home/psergey/dev-git/10.5-json/sql/sql_limit.h:67 (gdb) wher #0 0x0000555555d8ac12 in Select_limit_counters::get_select_limit (this=0xa5a5a5a5a5a5ad2d) at /home/psergey/dev-git/10.5-json/sql/sql_limit.h:67 #1 0x0000555555e2a4a0 in JOIN::optimize_inner (this=0x7fffac018520) at /home/psergey/dev-git/10.5-json/sql/sql_select.cc:1805 #2 0x0000555555e29b12 in JOIN::optimize (this=0x7fffac018520) at /home/psergey/dev-git/10.5-json/sql/sql_select.cc:1612 BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 1

[Maria-developers] OSX fix need on MariaDB-10.5
by Daniel Black 09 May '20

09 May '20

Mac lovers (who can use C/C++ and a debugger), As your MariaDB developers are hard at work getting 10.5 ready for you, unfortunately a bug has slipped into the 10.5 branch making it unusable on OSX (at least that's what it looks like on the CI infrastructure). The bug report is here: https://jira.mariadb.org/browse/MDEV-22173 As the time of developers is a little short as is the availability of OSX hardware, I'm hoping some of you are able to spend a few hours tackling this task to understand it better, and if possible, create patch. Steps: 1. build environment https://mariadb.com/kb/en/Build_Environment_Setup_for_Mac/ 2. clone the source git clone --branch 10.5 https://github.com/MariaDB/server.git mariadb-server cd mariadb-server git submodule update --recursive --init 3. build instructions https://mariadb.com/kb/en/generic-build-instructions/ (you don't need to install) 4. test (from build directory) mysql-test/mysql-test-run main.select might be useful here: --debugger=NAME or --gdb client-debugger=NAME or client-gdb To collaborative share your findings and to ask for further help MariaDB's community chat stream has been created for you. https://mariadb.zulipchat.com/#narrow/stream/118759-general/topic/Get.20OSX… To paraphrase J.F.Kennedy ask not what your database server community can do for you — ask what you can do for your database server community. Keep the community spirit going, Daniel Black -- Sent as a community member who noticed OSX failures and that everyone is busy

2 3

Re: [Maria-developers] 4312f382b6f: MDEV-22312: Bad error message for SET DEFAULT ROLE when user account is not granted the role
by Sergei Golubchik 08 May '20

08 May '20

Hi, Anel! On May 08, Anel Husakovic wrote: > revision-id: 4312f382b6f (mariadb-10.1.43-138-g4312f382b6f) > parent(s): 8c4b5261210 > author: Anel Husakovic <anel(a)mariadb.org> > committer: Anel Husakovic <anel(a)mariadb.org> > timestamp: 2020-05-07 16:49:33 +0200 > message: > > MDEV-22312: Bad error message for SET DEFAULT ROLE when user account is not granted the role > > diff --git a/mysql-test/suite/roles/set_role-recursive.test b/mysql-test/suite/roles/set_role-recursive.test > index 23d623e1966..dcf3dd5b6fc 100644 > --- a/mysql-test/suite/roles/set_role-recursive.test > +++ b/mysql-test/suite/roles/set_role-recursive.test > @@ -44,7 +44,7 @@ select * from mysql.roles_mapping; > > --sorted_result > show grants; > ---error ER_INVALID_ROLE > +--error ER_SET_DEFAULT_ROLE_FOR_USER > set role test_role2; > select current_user(), current_role(); > --sorted_result > diff --git a/sql/share/errmsg-utf8.txt b/sql/share/errmsg-utf8.txt > index c5e83062f0f..83ecd4735e8 100644 > --- a/sql/share/errmsg-utf8.txt > +++ b/sql/share/errmsg-utf8.txt > @@ -7144,3 +7144,5 @@ ER_WARN_AGGFUNC_DEPENDENCE > ukr "Агрегатна функція '%-.192s)' з SELECTу #%d належить до SELECTу #%d" > WARN_INNODB_PARTITION_OPTION_IGNORED > eng "<%-.64s> option ignored for InnoDB partition" > +ER_SET_DEFAULT_ROLE_FOR_USER > + eng "User '%s' has not been granted a role '%s'." Sorry, this is an absolute no-no. We cannot add new error messages to the old GA versions. Only to the very last RC/GA or to any alpha/beta. When you add an error message, all error messages added after it in newer versions will shift, change their numbers. And error numbers in RC/GA versions cannot change. Instead of a new error message and my_error(ER_SET_DEFAULT_ROLE_FOR_USER, MYF(0), c_usr.ptr(), rolename); you have to write my_printf_error(ER_INVALID_ROLE, "User %`s has not been granted a role %`s", MYF(0), c_usr.c_ptr(), rolename); note other changes: * %`s instead of '%s' for correct identifier quoting * no dot at the end * c_ptr() because printf expects a zero-terminated C string, and ptr() doesn't guarantee that. > diff --git a/sql/sql_acl.cc b/sql/sql_acl.cc > index b6a6f806e50..37141dd2390 100644 > --- a/sql/sql_acl.cc > +++ b/sql/sql_acl.cc > @@ -2041,6 +2041,9 @@ static int check_user_can_set_role(const char *user, const char *host, > ACL_USER_BASE *acl_user_base; > ACL_USER *UNINIT_VAR(acl_user); > bool is_granted= FALSE; > + bool user_can_see_role= FALSE; > + char buf[1024]; > + String c_usr(buf, sizeof(buf), &my_charset_bin); use StringBuffer<1024> c_str; > int result= 0; > > /* clear role privileges */ > @@ -2083,14 +2086,39 @@ static int check_user_can_set_role(const char *user, const char *host, > is_granted= TRUE; > break; > } > + > + for (uint i=0; i < acl_user->role_grants.elements; i++) > + { > + ACL_ROLE *r= *(dynamic_element(&acl_user->role_grants, i, ACL_ROLE**)); > + if (!strcmp(safe_str(r->user.str), safe_str(rolename))) > + { > + size_t len= acl_user->user.length + acl_user->hostname_length + 2; > + c_usr.alloc(len); > + const char c= '@'; > + strmov(strmov(strmov(const_cast<char*>(c_usr.ptr()), > + safe_str(user)), &c), safe_str(host)); > + user_can_see_role= TRUE; > + } > + } No, you repeat this for() block for evert role grant of every role's grantee. And you only use it when there was an error, an uncommon case. Do not do the work that you aren't going to use. And the check doesn't seem correct. You want to use the same rule as for applicable_roles, you need to traverse the graph. Perhaps you can simply use check_role_is_granted() though. And also you need to check whether the current user has read access on mysql.user table - in that case it can see all roles, granted or not. > } > > /* According to SQL standard, the same error message must be presented */ this comment now belongs inside the if(), above ER_INVALID_ROLE > if (!is_granted) > { > - my_error(ER_INVALID_ROLE, MYF(0), rolename); > - result= 1; > - goto end; > + /* Handling of the bad error message - MDEV-22312 */ don't put MDEV numbers into the code, please. > + /* If the current user can see the role generate different type of error. */ > + if(user_can_see_role) > + { > + my_error(ER_SET_DEFAULT_ROLE_FOR_USER, MYF(0), c_usr.ptr(), rolename); > + result= 1; > + goto end; > + } > + else > + { > + my_error(ER_INVALID_ROLE, MYF(0), rolename); > + result= 1; > + goto end; > + } > } > > if (access) Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

1 0

[Maria-developers] Welcome to Google Summer of Code 2020 with MariaDB
by Vicențiu Ciorbaru 04 May '20

04 May '20

Hi! GSoC 2020 accepted proposals have just been announced and you are part of the successful applicants. This year we had a lot of very good proposals and so we have a one of the highest number of students (7) to mentor for GSoC. Congratulations on the effort to learn about the program, get involved and submit successful proposals! The fun part begins now! If you have not done so already, please reach out to your mentor and seek further advice about your project. Each one of your projects can be implemented in multiple ways, some more complex than others. I strongly advise you to create a concrete plan based off of your proposal with your mentor and aim to stick to it. The MariaDB Server codebase is a complex one with lots of different parts, however your mentor can guide you to ensure you have a successful GSoC. Remember that you are here to learn about Open Source, what it means to be part of an Open Source community, to interact with the community, all while becoming better programmers. Please take advantage of this opportunity and keep your technical discussions public, on Zulip https://mariadb.zulipchat.com, and on our developer mailing list https://launchpad.net/~maria-developers. By keeping your discussions public, you get the benefit of not only having your mentor see your questions but all of the MariaDB community. You never know when someone might lend a hand in case your mentor is not yet online. :) The mantra of our community should be that we are always welcoming to new people. Do not be afraid to ask questions. There are no bad or stupid or silly questions. The only key requirement is that you first make an effort to find the answer yourself and only afterwards seek outside help. If you can show a bit of involvement, the community will show you the same. I recommend that you agree with your mentor on some form of weekly progress reporting, via either a blog-post, an email, or something similar that can be publicly shared. I can not stress this enough: we are working on an open source project, our work must be done in the open and it is important to *show the world the good we are doing*. Hope all of you will have a great GSoC! Vicențiu MariaDB Foundation GSoC Admin

1 0