July 2018 - developers - lists.mariadb.org

[Maria-developers] Windows line endings in MariaDB sources again (
by Otto Kekäläinen 24 Jul '19

24 Jul '19

Hello! While trying to release 10.0.26 for Debian and Ubuntu I noticed git chokes again on Windows line endings. As our .gitattributes states that by default git should clean up the line endings, why do we keep getting new Windows line endings in new code? mariadb-10.0$ cat .gitattributes # Set the default behavior, in case people don't have core.autocrlf set. * text=auto ... storage/connect/mysql-test/connect/std_data/*.txt -text mariadb-10.0.26$ find . | xargs file | grep CRLF ./storage/connect/mysql-test/connect/std_data/boyswin.txt: ASCII text, with CRLF line terminators ./storage/connect/mysql-test/connect/std_data/expenses.txt: ASCII text, with CRLF line terminators ./storage/connect/mysql-test/connect/std_data/emp.txt: ASCII text, with CRLF line terminators ./mysql-test/std_data/loaddata7.dat: ASCII text, with CRLF line terminators ./mysql-test/r/loadxml.result: ASCII text, with CRLF, LF line terminators ./mysql-test/r/perror-win.result: ASCII text, with CRLF, LF line terminators ./mysql-test/r/mysql_binary_mode.result: ASCII English text, with CRLF, LF line terminators ./mysql-test/r/func_regexp_pcre.result: UTF-8 Unicode C++ program text, with CRLF, CR, LF line terminators ./pcre/testdata/grepoutputN: ASCII text, with CRLF, CR, LF line terminators ./pcre/testdata/greppatN4: ASCII text, with CRLF line terminators This leads for me to: dpkg-source: info: building mariadb-10.0 using existing ./mariadb-10.0_10.0.26.orig.tar.gz dpkg-source: info: local changes detected, the modified files are: mariadb-10.0/storage/connect/mysql-test/connect/std_data/boyswin.txt mariadb-10.0/storage/connect/mysql-test/connect/std_data/emp.txt mariadb-10.0/storage/connect/mysql-test/connect/std_data/expenses.txt dpkg-source: error: aborting due to unexpected upstream changes, see /tmp/mariadb-10.0_10.0.26-1.diff.Yzwwhw Which means I have to do extra work with importing the upstream sources.. - Otto

2 4

[Maria-developers] 10.2 and merges (e.g. TokuDB)
by Phil Sweeney 28 Aug '18

28 Aug '18

Hi there, It seems 10.2.6 is still shipping with TokuDB from PS 5.6.35-80 (Feb 8th 2017). I can't see a jira issue for a 10.2 merge since 10.2.4. Perhaps intentional to help stabilise 10.2 for release? 10.0.30 also shipped with the same Feb version (in March). 10.1.23 however was released in May and still has this version from Feb, which seems unfortunate, but I know it's because it has to flow through 10.0 first. I see 10.0.31 is due to get TokuDB from PS 5.6.36-82 And that should flow into 10.1.24 soon too. However, 10.2.7 isn't due until July, so by then TokuDB version will be 5 months old in 10.2 I'm waiting on a fairly important row count fix that should appear in the next Percona Server release. When it does, if the current merging strategy continues, do I have to first wait for 10.0.32 to get it in late July, then 10.1.26 in August, then 10.2 after that? Or with 10.0, 10.1 and 10.2 all being supported and needing merges, is there a new strategy on the cards to shorten this lag? Thanks! Phil

3 7

[Maria-developers] Spelling errors in MariaDB sources
by Otto Kekäläinen 31 Jul '18

31 Jul '18

Hello! I recall I've made several big patches on MariaDB that fixes spelling errors. Somehow the same spelling errors seem to creep back in every now and then. I notice them while uploading stuff to Debian and using the Debian QA tool "Lintian". I haven't done git blame and looked deeply in it. I suspect it is related to how we merge old branches to new branches and occasionally loose stuff. Would somebody like to look into this? It could be an relatively easy task for some new contributor? $ grep spelling-error lintian-22e620c534.log I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_chk noone no one I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_chk thats that's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_chk theres there's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_dump_log noone no one I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_dump_log thats that's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_dump_log theres there's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_ftdump noone no one I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_ftdump thats that's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_ftdump theres there's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_pack noone no one I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_pack thats that's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_pack theres there's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_read_log noone no one I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_read_log thats that's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/aria_read_log theres there's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisam_ftdump noone no one I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisam_ftdump thats that's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisam_ftdump theres there's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisamchk noone no one I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisamchk thats that's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisamchk theres there's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisamlog noone no one I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisamlog thats that's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisamlog theres there's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisampack noone no one I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisampack thats that's I: mariadb-server-10.1: spelling-error-in-binary usr/bin/myisampack theres there's I: mariadb-server-10.1: spelling-error-in-binary usr/lib/x86_64-linux-gnu/mariadb18/plugin/ha_federated.so Allows to Allows one to I: mariadb-server-10.1: spelling-error-in-binary usr/lib/x86_64-linux-gnu/mariadb18/plugin/ha_federatedx.so Allows to Allows one to I: mariadb-server-10.1: spelling-error-in-binary usr/lib/x86_64-linux-gnu/mariadb18/plugin/ha_innodb.so allow to allow one to I: libmariadbd18: spelling-error-in-binary usr/lib/x86_64-linux-gnu/libmysqld.so.18 extentions extensions I: libmariadbd18: spelling-error-in-binary usr/lib/x86_64-linux-gnu/libmysqld.so.18 noone no one I: libmariadbd18: spelling-error-in-binary usr/lib/x86_64-linux-gnu/libmysqld.so.18 thats that's I: libmariadbd18: spelling-error-in-binary usr/lib/x86_64-linux-gnu/libmysqld.so.18 theres there's I: libmariadbd18: spelling-error-in-binary usr/lib/x86_64-linux-gnu/libmysqld.so.18 allow to allow one to I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup yuR your I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup occured occurred I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup accessable accessible I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup extentions extensions I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup reseting resetting I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup succeded succeeded I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup settting setting I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup postion position I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup noone no one I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup thats that's I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup theres there's I: mariadb-client-10.1: spelling-error-in-binary usr/bin/mariabackup allow to allow one to I: mariadb-client-10.1: spelling-error-in-manpage usr/share/man/man1/innotop.1.gz Additonal Additional I: mariadb-plugin-tokudb: spelling-error-in-binary usr/bin/tokuftdump dont don't I: mariadb-plugin-tokudb: spelling-error-in-binary usr/lib/x86_64-linux-gnu/mariadb18/plugin/ha_tokudb.so dont don't I: mariadb-client-core-10.1: spelling-error-in-binary usr/bin/mysql_embedded extentions extensions I: mariadb-client-core-10.1: spelling-error-in-binary usr/bin/mysql_embedded noone no one I: mariadb-client-core-10.1: spelling-error-in-binary usr/bin/mysql_embedded thats that's I: mariadb-client-core-10.1: spelling-error-in-binary usr/bin/mysql_embedded theres there's I: mariadb-client-core-10.1: spelling-error-in-binary usr/bin/mysql_embedded allow to allow one to I: mariadb-server-core-10.1: spelling-error-in-binary usr/sbin/mysqld extentions extensions I: mariadb-server-core-10.1: spelling-error-in-binary usr/sbin/mysqld reseting resetting I: mariadb-server-core-10.1: spelling-error-in-binary usr/sbin/mysqld succeded succeeded I: mariadb-server-core-10.1: spelling-error-in-binary usr/sbin/mysqld settting setting I: mariadb-server-core-10.1: spelling-error-in-binary usr/sbin/mysqld postion position I: mariadb-server-core-10.1: spelling-error-in-binary usr/sbin/mysqld noone no one I: mariadb-server-core-10.1: spelling-error-in-binary usr/sbin/mysqld thats that's I: mariadb-server-core-10.1: spelling-error-in-binary usr/sbin/mysqld theres there's I: mariadb-server-core-10.1: spelling-error-in-binary usr/sbin/mysqld allow to allow one to I: mariadb-plugin-connect: spelling-error-in-binary usr/lib/x86_64-linux-gnu/mariadb18/plugin/ha_connect.so Mutiple Multiple I: mariadb-plugin-connect: spelling-error-in-binary usr/lib/x86_64-linux-gnu/mariadb18/plugin/ha_connect.so Openning Opening

1 0

[Maria-developers] Missing man pages
by Otto Kekäläinen 31 Jul '18

31 Jul '18

Hello Ian and others! While working on uploading latest MariaDB 10.1 into Debian I noticed it complains about quite a few missing man pages: $ grep manpage lintian-22e620c534.log W: mariadb-server-10.1: binary-without-manpage usr/bin/wsrep_sst_mariabackup W: mariadb-server-10.1: binary-without-manpage usr/bin/wsrep_sst_rsync_wan I: mariadb-client-10.1: spelling-error-in-manpage usr/share/man/man1/innotop.1.gz Additonal Additional W: mariadb-client-10.1: binary-without-manpage usr/bin/mariabackup W: mariadb-client-10.1: binary-without-manpage usr/bin/mbstream W: mariadb-client-10.1: binary-without-manpage usr/bin/mytop W: mariadb-plugin-tokudb: binary-without-manpage usr/bin/tokuft_logprint W: mariadb-client-core-10.1: binary-without-manpage usr/bin/mysql_embedded Could you Ian check if we have these on some branch already and make sure they and up an all the same branches and releases as the binaries themselves? And those who make new binaries, please at least write a stub for your new binary. It makes downstream work much faster when the man page in the source explains directly what the new binaries are and hints how they should be packaged. - Otto

1 0

[Maria-developers] Please review MDEV-16584 SP with a cursor inside a loop wastes THD memory aggressively
by Alexander Barkov 30 Jul '18

30 Jul '18

Hello Sanja, Please review a patch for MDEV-16584. Thanks!

2 1

Re: [Maria-developers] [Commits] 108af298032: MDEV-15454: Nested SELECT IN returns wrong results
by Sergey Petrunia 25 Jul '18

25 Jul '18

On Fri, May 11, 2018 at 03:08:29AM +0530, Varun wrote: > revision-id: 108af298032cb4cea316a08c304c306268a2badf (mariadb-5.5.56-213-g108af298032) > parent(s): 318097bb8f6e12c546b5dcd287416158209dbb39 > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-05-11 02:56:33 +0530 > message: > > MDEV-15454: Nested SELECT IN returns wrong results > > In this case we are setting the field Item_func_eq::in_eqaulity_no for the semi-join equalities. > This helps us to remove these equalites as the inner tables are not available during parent select execution > while the outer tables are not available during materialization phase. > We only have it set for the equalites for the fields involved with the IN subquery > and reset it for the equalities which do not belong to the IN subquery. > > For example in case of nested IN subqueries: > > SELECT t1.a FROM t1 WHERE t1.a IN > (SELECT t2.a FROM t2 where t2.b IN > (select t3.b from t3 where t3.c=27 )) > > there are two equalites involving the fields of the IN subquery > > 1) t2.b = t3.b : the field Item_func_eq::in_eqaulity_no is set when we merge the grandchild select into the child select > 2) t1.a = t2.a : the field Item_func_eq::in_eqaulity_no is set when we merge the child select into the parent select > > But when we perform case 2) we should ensure that we reset the equalities in the child's WHERE clause. > > --- > mysql-test/r/subselect_sj_mat.result | 46 +++++++++++++++++++++++++ > mysql-test/t/subselect_sj_mat.test | 49 ++++++++++++++++++++++++++ > sql/opt_subselect.cc | 66 ++++++++++++++++++++++++++++++++++++ > 3 files changed, 161 insertions(+) > ... > diff --git a/sql/opt_subselect.cc b/sql/opt_subselect.cc > index a7edd64e68b..31213a5cf1d 100644 > --- a/sql/opt_subselect.cc > +++ b/sql/opt_subselect.cc > @@ -1481,6 +1543,7 @@ static int subq_sj_candidate_cmp(Item_in_subselect* el1, Item_in_subselect* el2, > TRUE Out of memory error > */ > > + > static bool convert_subq_to_sj(JOIN *parent_join, Item_in_subselect *subq_pred) > { > SELECT_LEX *parent_lex= parent_join->select_lex; > @@ -1713,6 +1776,7 @@ static bool convert_subq_to_sj(JOIN *parent_join, Item_in_subselect *subq_pred) > > if (subq_pred->left_expr->cols() == 1) > { > + reset_equality_number_for_subq_conds(sj_nest->sj_on_expr); > /* add left = select_list_element */ > nested_join->sj_outer_expr_list.push_back(&subq_pred->left_expr); > /* > @@ -1735,6 +1799,7 @@ static bool convert_subq_to_sj(JOIN *parent_join, Item_in_subselect *subq_pred) > } > else if (subq_pred->left_expr->type() == Item::ROW_ITEM) > { > + reset_equality_number_for_subq_conds(sj_nest->sj_on_expr); > /* > disassemple left expression and add > left1 = select_list_element1 and left2 = select_list_element2 ... > @@ -1759,6 +1824,7 @@ static bool convert_subq_to_sj(JOIN *parent_join, Item_in_subselect *subq_pred) > } > else > { > + reset_equality_number_for_subq_conds(sj_nest->sj_on_expr); > /* > add row operation > left = (select_list_element1, select_list_element2, ...) So, the code here is if (...) { variant1; } else if (...) { variant2; } else { variant3; } and there is the same call added to all three: > + reset_equality_number_for_subq_conds(sj_nest->sj_on_expr); Can you move it out of the "if" and make it one call instead of three? Otherwise, the patch is ok, ok to push after the above is addressed. BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] d277c4682e4: MDEV-16751: Server crashes in st_join_table::cleanup or TABLE_LIST::is_with_table_recursive_reference
by Sergey Petrunia 24 Jul '18

24 Jul '18

On Sun, Jul 22, 2018 at 03:56:37PM +0530, Varun wrote: > revision-id: d277c4682e4c6bb0e708296e6ae520c149dd78f4 (mariadb-5.5.60-47-gd277c4682e4) > parent(s): 9cea4ccf12cb6e8746b9b440d9c62408a9ef04af > author: Varun Gupta > committer: Varun Gupta > timestamp: 2018-07-22 15:56:25 +0530 > message: > > MDEV-16751: Server crashes in st_join_table::cleanup or TABLE_LIST::is_with_table_recursive_reference > with join_cache_level>2 > > During muliple equality propagation for a query in which we have an IN subquery, the items in the select list of the > subquery may not be part of the multiple equality because there might be another occurence of the same field in the > where clause of the subquery. > So we keyuse_is_valid_for_access_in_chosen_plan function which expects the items in the select list of the subquery to > be same to the ones in the multiple equality (through these multiple equalities we create keyuse array). > The solution would be that we expect the same field not the same Item because when we have SEMI JOIN MATERIALIZATION SCAN, > we use copy back technique to copies back the materialised table fields to the original fields of the base tables. > > --- > mysql-test/r/subselect_mat.result | 35 +++++++++++++++++++++++++++++++++++ > mysql-test/r/subselect_sj_mat.result | 35 +++++++++++++++++++++++++++++++++++ > mysql-test/t/subselect_sj_mat.test | 25 +++++++++++++++++++++++++ > sql/sql_select.cc | 4 +++- > 4 files changed, 98 insertions(+), 1 deletion(-) ... > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index 6a64a0e9952..af6f63a55f1 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -7438,7 +7438,9 @@ bool JOIN_TAB::keyuse_is_valid_for_access_in_chosen_plan(JOIN *join, > st_select_lex *sjm_sel= emb_sj_nest->sj_subq_pred->unit->first_select(); > for (uint i= 0; i < sjm_sel->item_list.elements; i++) > { > - if (sjm_sel->ref_pointer_array[i] == keyuse->val) > + DBUG_ASSERT(keyuse->val->type() == Item::FIELD_ITEM); ^^(1) > + Field *field = ((Item_field*)sjm_sel->ref_pointer_array[i])->field; ^^(2) > + if (field->eq(((Item_field*)keyuse->val)->field)) > return true; > } > return false; As for typecast (2): As far as I understand, it is valid because of the following: - We get to this point in this function only when Semi-Join-Materialization-Scan is used. - SJM-Scan is only used when the subquery's select list has Item_field objects (This is checked in subquery_types_allow_materialization()). * Please add a comment in subquery_types_allow_materialization() about this function relying on that logic. * Please add here an assert that sjm_sel->ref_pointer_array[i] is an Item_field. The logic is complex and can be potentially broken. As for typecast (1): What if keyuse->val is not an Item_field? If we get to this point in this function, it means it's an expression that only depends on the SJ-Inner table. This cannot just happen due to limitations in subquery_types_allow_materialization(), and I wasn't able to construct an example where it would happen (tried playing with multiple-equalities, but substitute_for_best_equal_item is called after get_best_combination(), tried playing with constant tables but then SJ-Materialization is not used, etc. ) I think we will be safer by changing the assert to be this check: if (keyuse->val->type() == Item::FIELD_ITEM) { ... everything else } The idea is that a KEYUSE that satisfies the requirement will still be available, and we just want to filter out any extra KEYUSE that might pop up due to some optimization. what do you think? Ok to push after the above is addressed. BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] Updated Gtid_slave_pos of Untracked domain creates skipped events
by Sachin Setiya 23 Jul '18

23 Jul '18

Hi Andrei, Kristian! Problem:- This issue is regarding Mdev-9107 , where we have 3 master master with do_domain_ids of replication channel master domain-id . , When one replication channel is stopped and that master is updated , slave server updates the gtid slave_pos from second replication channel. And then when replication channel is brought up again then since we have updated the gtid_slave_pos we skip that event. I have created a abstract test case for this probem. (3) ^ ^ / \ (2)----->(1) 3 and 1 is configured with log slave updates. 3 has 2 replication channel m2_s3(do_domain_id=2 ), m1_s3(do_domain_id=1) 1 has one replication channel m2_s1(do_domain_id=2) I have tried with master_use_gtid == slave_pos and current_pos both give same result. When I stop replication channel m2_s3 and then write one-one event in (2) and (1) server. Server 3 will update its gtid_slave_pos and it will have latest gtid event seq no from domain id 2 and 1. When replication channel m2_s3 is bought up again since gtid_slave_pos of (3) is equal to (2) so we actually skipped the event generated on server 2. (I believe network lag between 2-->3 will have similar effect) (I have attached the test cases) Possible Solution. May be we should have one more option in master_use_gtid = binlog_state ? which will compare its binlog_state to arriving event gtid no ? (It is different from current_pos , current_pos will select the maximum seq_no between binlog_state and slave_pos) -- Regards Sachin Setiya Software Engineer at MariaDB

2 5

[Maria-developers] Please review a patch for MDEV-15758 Split Item_bool_func::get_mm_leaf() into virtual methods in Field and Type_handler
by Alexander Barkov 20 Jul '18

20 Jul '18

Hello Vicentiu, Please review my patch for MDEV-15758. It also fixes this bug: MDEV-15759 Expect "Impossible WHERE" for indexed_int_column=out_of_range_int_constant Thanks!

3 4

[Maria-developers] To MDEV-6608 Slave loses master binlog filename
by andrei.elkin＠pp.inet.fi 13 Jul '18

13 Jul '18

Hello, Kristian. I read through your analysis of the case to agree with the plan and its motivation: MDEV-6608> Generally, it is a very bad idea to destroy data during crash recovery ... try an approach that leaves the crashed binlog file intact, and instead ensures that the crashed binlog can be properly handled by the rest of the code. So in case of a partially written event group (transaction) the short (xid-less therefore) transaction is to be rolled back on the master. So why won't we properly enclose the group in the binlog at the end of recovery which means to append ROLLBACK Query-log-event, naturally before that the last event of the group needs sanity check? Was that that you had in mind actually? Cheers, Andrei

2 1