- developers - lists.mariadb.org

[Maria-developers] implementing index condition pushdown in MariaDB 5.5
by Zardosht Kasheff 16 Feb '13

16 Feb '13

Hello, Is there any documentation for what a storage engine needs to do to implement index condition pushdown in MariaDB 5.5? I see some related things, such as handler_index_cond_check, HA_DO_INDEX_COND_PUSHDOWN, and idx_cond_push, but I don't understand how these all interact with each other. Thanks -Zardosht

2 6

[Maria-developers] Help with running a transaction inside the server
by Kristian Nielsen 14 Feb '13

14 Feb '13

Hi Serg, Can you give me a tip for doing the following (for MDEV-26, global transaction ID): I need to run a transaction inside the server. This is in the slave I/O thread, when it connects to the master it receives the current replication position as a GTID, and I want to store it in the mysql.rpl_slave_state so that next reconnect of the slave can use the GTID automatically. So I basically need to run the equivalent of something like this: BEGIN; INSERT INTO mysql.rpl_slave_state VALUES (...); DELETE FROM mysql.rpl_slave_state WHERE id=...; COMMIT; I think I already have everything except the "BEGIN" part (if you want, you can check the code in rpl_slave_state::record_gtid() in my branch lp:~maria-captains/maria/10.0-mdev26/ on Launchpad). So how do I do the equivalent of BEGIN inside the server code? Can you think of any existing code I could use as example? - Kristian.

3 7

[Maria-developers] Fwd: debugging how query plans are made
by Zardosht Kasheff 12 Feb '13

12 Feb '13

Is there something similar in MariaDB? ---------- Forwarded message ---------- From: Øystein Grøvlen <oystein.grovlen(a)oracle.com> Date: Tue, Feb 12, 2013 at 9:11 AM Subject: Re: debugging how query plans are made To: Zardosht Kasheff <zardosht(a)gmail.com> Cc: "internals(a)lists.mysql.com" <internals(a)lists.mysql.com> On 02/12/13 02:54 PM, Zardosht Kasheff wrote: > > Hello all, > > In MySQL and/or MariaDB, are there any hooks or methods one can use to > learn why a query plan is chosen over alternatives? We sometimes > notice that a different query plan is chosen for us than InnoDB (which > may be fine, given we are a different storage engine), and we would > like to better understand why. Is there any debug parameter that gets > MySQL to output decisions made in the optimizer? MySQL 5.6 has optimizer tracing which outputs the decisions made by the optimizer. See http://dev.mysql.com/doc/internals/en/optimizer-tracing.html Hope this helps, -- Øystein

2 1

Re: [Maria-developers] Files under /etc/my.cnf.d/ read by default
by Sergei Golubchik 12 Feb '13

12 Feb '13

Hi, Honza! On Feb 05, Honza Horak wrote: > Hi guys, > > while we were packaging MariaDB into Fedora (finally) we used the > upstream's cnf files structure, which means /etc/my.cnf, that includes > all files from directory /etc/my.cnf.d/* using statement !includedir. > > That works fine when user installs (not updates) the MariaDB packages > without having them installed before. The problem is if user has changed > /etc/my.cnf before updating -- then /etc/my.cnf doesn't get updated > (!includedir won't be added) and we end with files under /etc/my.cnf.d/* > but nothing what includes them. That is quite confusing for users that > may change something in /etc/my.cnf.d/client.cnf but nothing happens. Agree. Especially as we've started to use /etc/my.cnf.d/ now - e.g. CassandraSE comes in a separate rpm, that (besides the plugin itself) drops a file into /etc/my.cnf.d/ to enable the engine in the server. I expect that more separately-packed plugins will start doing the same. > Since files under /etc/my.cnf.d/* are empty by default it shouldn't do > any harm if all files corresponding with /etc/my.cnf.d/*.cnf template > get loaded even without !includedir in /etc/my.cnf. Actually I believe > that is what users expect to happen when they see files under > /etc/my.cnf.d.. > > So, that's basically what I'm proposing -- read files under > /etc/my.cnf.d/* by default, without a need to use !includedir. What do > you think? What if we add !includedir to the /etc/my.cnf from the post-inst script? Something like grep -q '!includedir /etc/my.cnf.d' /etc/my.cnf || \ (echo; echo '!includedir /etc/my.cnf.d') >> /etc/my.cnf Would that be enough? Regards, Sergei

2 1

[Maria-developers] No Doxygen?
by Christian Convey 11 Feb '13

11 Feb '13

I was kind of surprised to not find anything in MariaDB's build system for using Doxygen to create HTML docs. Have people decided this isn't a worthwhile thing to do? Or has just no one gotten around to it yet? - Christian

3 4

Re: [Maria-developers] Question
by Michael Widenius 10 Feb '13

10 Feb '13

Hi! First, the MariaDB developers list is the right place to ask questions like this. >>>>> "Aamir" == Aamir Abbas <Abbas> writes: Aamir> Hi Monty, Aamir> I have researched this question about MySQL without luck and if you cannot answer it then nobody can. On some MySQL servers the version number returns a "-log" suffixed to the version number. For example, 5.0.45-log. What does the "-log" mean or signify? This is not a standard version number as far as I can tell. Should it not simply return 5.0.45? I have posted the question to a couple of forums but nobody's able to answer. I have just updated the documentation at https://kb.askmonty.org/en/version/ to answer this questions in detail. Here is a copy of the new documentation: The VERSION() string may have one or more of the following suffixes: Suffix Description -embedded The server is an embedded server (libmysqld). -log General logging, slow logging or binary (replication) logging is enabled. -debug The server is compiled for debugging. -valgrind The server is compiled to be instrumented with valgrind. Regards, Monty

1 0

[Maria-developers] Coverity scan results
by Christian Convey 07 Feb '13

07 Feb '13

Hey guys, For those of you who missed it, I volunteered to try gettin MariaDB scanned as part of the Coverity Scan service. I just got the trunk scanned for the first time, and a lot of potential-problem reports came up. I did a spot check on one of them, and the report looks like a true-positive. Coverity Scan reports: 178 high-impact problems 1020 medium-impact problems 47 low-impact problems. Any suggestions for how to get these looked at by the appropriate developers? It seems crazy for me to enter 1000+ bug reports into the bug tracker. I believe I have the power to create new user accounts in Coverity Scan's MariaDB project, for any MariaDB developer who wants to see the results. Thanks, Christian

3 12

Re: [Maria-developers] Fwd: [Commits] Rev 3622: Fix for bug MDEV-765 (LP:825075) in file:///home/tsk/mprog/src/5.3-md765/
by Sergei Petrunia 07 Feb '13

07 Feb '13

Hi Timour, On Mon, Feb 04, 2013 at 05:39:02PM +0200, Timour Katchaounov wrote: > > Sergey, > > Could you please review the following patch. The fix implements > your suggestion from your previous review: > https://lists.launchpad.net/maria-developers/msg04597.html > Ok to push. > Timour > > ------------------------------------------------------------ > revno: 3622 > revision-id: timour(a)askmonty.org-20130204153548-njv08hcdskv6ttjk > parent: sergii(a)pisem.net-20130128081223-mp9rsd3t9soz8lly > fixes bug: https://mariadb.atlassian.net/browse/MDEV-765 > committer: timour(a)askmonty.org > branch nick: 5.3-md765 > timestamp: Mon 2013-02-04 17:35:48 +0200 > message: > Fix for bug MDEV-765 (LP:825075) > > Analys: > The cause for the wrong result was that the optimizer > incorrectly chose min/max loose scan when it is not > applicable. The applicability test missed the case when > a condition on the MIN/MAX argument was OR-ed with a > condition on some other field. In this case, the MIN/MAX > condition cannot be used for loose scan. > > Solution: > Extend the test check_group_min_max_predicates() to check > that the WHERE clause is of the form: "cond1 AND cond2" > where > cond1 - does not use min_max_column at all. > cond2 - is an AND/OR tree with leaves in form "min_max_column $CMP$ const" > or $CMP$ is one of the functions between, is [not] null > > > === modified file 'mysql-test/r/group_min_max.result' > --- a/mysql-test/r/group_min_max.result 2013-01-28 08:12:23 +0000 > +++ b/mysql-test/r/group_min_max.result 2013-02-04 15:35:48 +0000 > @@ -3186,3 +3186,61 @@ a b > 109 19 > drop table t1; > End of 5.1 tests > +# > +# MDEV-765: LP:825075 - Wrong result with GROUP BY + multipart key + MIN/MAX loose scan > +# > +CREATE TABLE t1 (a varchar(1), b varchar(1), KEY (b,a)); > +INSERT INTO t1 VALUES > +('0',NULL),('9',NULL),('8','c'),('4','d'),('7','d'),(NULL,'f'), > +('7','f'),('8','g'),(NULL,'j'); > +explain > +SELECT max(a) , b FROM t1 WHERE a IS NULL OR b = 'z' GROUP BY b; > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE t1 index b b 8 NULL 9 Using where; Using index > +SELECT max(a) , b FROM t1 WHERE a IS NULL OR b = 'z' GROUP BY b; > +max(a) b > +NULL f > +NULL j > +explain > +SELECT b, min(a) FROM t1 WHERE a = '7' OR b = 'z' GROUP BY b; > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE t1 index b b 8 NULL 9 Using where; Using index > +SELECT b, min(a) FROM t1 WHERE a = '7' OR b = 'z' GROUP BY b; > +b min(a) > +d 7 > +f 7 > +explain > +SELECT b, min(a) FROM t1 WHERE (a = b OR b = 'd' OR b is NULL) GROUP BY b; > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE t1 index b b 8 NULL 9 Using where; Using index > +SELECT b, min(a) FROM t1 WHERE (a = b OR b = 'd' OR b is NULL) GROUP BY b; > +b min(a) > +NULL 0 > +d 4 > +explain > +SELECT b, min(a) FROM t1 WHERE a > ('0' = b) AND b = 'z' GROUP BY b; > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE t1 ref b b 4 const 1 Using where; Using index > +SELECT b, min(a) FROM t1 WHERE a > ('0' = b) AND b = 'z' GROUP BY b; > +b min(a) > +explain > +SELECT b, min(a) FROM t1 WHERE a > '0' AND (b < (a = '7')) GROUP BY b; > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE t1 index NULL b 8 NULL 9 Using where; Using index > +SELECT b, min(a) FROM t1 WHERE a > '0' AND (b < (a = '7')) GROUP BY b; > +b min(a) > +d 7 > +f 7 > +explain > +SELECT b, min(a) FROM t1 WHERE (a > '0' AND (a > '1' OR b = 'd')) GROUP BY b; > +id select_type table type possible_keys key key_len ref rows Extra > +1 SIMPLE t1 index b b 8 NULL 9 Using where; Using index > +SELECT b, min(a) FROM t1 WHERE (a > '0' AND (a > '1' OR b = 'd')) GROUP BY b; > +b min(a) > +NULL 9 > +c 8 > +d 4 > +f 7 > +g 8 > +drop table t1; > +End of 5.3 tests > > === modified file 'mysql-test/t/group_min_max.test' > --- a/mysql-test/t/group_min_max.test 2013-01-28 08:12:23 +0000 > +++ b/mysql-test/t/group_min_max.test 2013-02-04 15:35:48 +0000 > @@ -1203,3 +1203,40 @@ WHERE EXISTS ( SELECT DISTINCT b FROM t1 > drop table t1; > > --echo End of 5.1 tests > + > +--echo # > +--echo # MDEV-765: LP:825075 - Wrong result with GROUP BY + multipart key + MIN/MAX loose scan > +--echo # > + > +CREATE TABLE t1 (a varchar(1), b varchar(1), KEY (b,a)); > +INSERT INTO t1 VALUES > +('0',NULL),('9',NULL),('8','c'),('4','d'),('7','d'),(NULL,'f'), > +('7','f'),('8','g'),(NULL,'j'); > + > +explain > +SELECT max(a) , b FROM t1 WHERE a IS NULL OR b = 'z' GROUP BY b; > +SELECT max(a) , b FROM t1 WHERE a IS NULL OR b = 'z' GROUP BY b; > + > +explain > +SELECT b, min(a) FROM t1 WHERE a = '7' OR b = 'z' GROUP BY b; > +SELECT b, min(a) FROM t1 WHERE a = '7' OR b = 'z' GROUP BY b; > + > +explain > +SELECT b, min(a) FROM t1 WHERE (a = b OR b = 'd' OR b is NULL) GROUP BY b; > +SELECT b, min(a) FROM t1 WHERE (a = b OR b = 'd' OR b is NULL) GROUP BY b; > + > +explain > +SELECT b, min(a) FROM t1 WHERE a > ('0' = b) AND b = 'z' GROUP BY b; > +SELECT b, min(a) FROM t1 WHERE a > ('0' = b) AND b = 'z' GROUP BY b; > + > +explain > +SELECT b, min(a) FROM t1 WHERE a > '0' AND (b < (a = '7')) GROUP BY b; > +SELECT b, min(a) FROM t1 WHERE a > '0' AND (b < (a = '7')) GROUP BY b; > + > +explain > +SELECT b, min(a) FROM t1 WHERE (a > '0' AND (a > '1' OR b = 'd')) GROUP BY b; > +SELECT b, min(a) FROM t1 WHERE (a > '0' AND (a > '1' OR b = 'd')) GROUP BY b; > + > +drop table t1; > + > +--echo End of 5.3 tests > > === modified file 'sql/opt_range.cc' > --- a/sql/opt_range.cc 2013-01-28 08:12:23 +0000 > +++ b/sql/opt_range.cc 2013-02-04 15:35:48 +0000 > @@ -11485,7 +11485,8 @@ static bool get_constant_key_infix(KEY * > KEY_PART_INFO **first_non_infix_part); > static bool > check_group_min_max_predicates(Item *cond, Item_field *min_max_arg_item, > - Field::imagetype image_type); > + Field::imagetype image_type, > + bool *has_min_max_fld, bool *has_other_fld); > > static void > cost_group_min_max(TABLE* table, KEY *index_info, uint used_key_parts, > @@ -12005,10 +12006,12 @@ get_best_group_min_max(PARAM *param, SEL > DBUG_RETURN(NULL); > > /* Check (SA3) for the where clause. */ > + bool has_min_max_fld= false, has_other_fld= false; > if (join->conds && min_max_arg_item && > !check_group_min_max_predicates(join->conds, min_max_arg_item, > (index_info->flags & HA_SPATIAL) ? > - Field::itMBR : Field::itRAW)) > + Field::itMBR : Field::itRAW, > + &has_min_max_fld, &has_other_fld)) > DBUG_RETURN(NULL); > > /* The query passes all tests, so construct a new TRP object. */ > @@ -12043,16 +12046,24 @@ get_best_group_min_max(PARAM *param, SEL > > SYNOPSIS > check_group_min_max_predicates() > - cond tree (or subtree) describing all or part of the WHERE > - clause being analyzed > - min_max_arg_item the field referenced by the MIN/MAX function(s) > - min_max_arg_part the keypart of the MIN/MAX argument if any > + cond [in] the expression tree being analyzed > + min_max_arg [in] the field referenced by the MIN/MAX function(s) > + image_type [in] > + has_min_max_arg [out] true if the subtree being analyzed references min_max_arg > + has_other_arg [out] true if the subtree being analyzed references a column > + other min_max_arg > > DESCRIPTION > The function walks recursively over the cond tree representing a WHERE > clause, and checks condition (SA3) - if a field is referenced by a MIN/MAX > aggregate function, it is referenced only by one of the following > - predicates: {=, !=, <, <=, >, >=, between, is null, is not null}. > + predicates $FUNC$: > + {=, !=, <, <=, >, >=, between, is [not] null, multiple equal}. > + In addition the function checks that the WHERE condition is equivalent to > + "cond1 AND cond2" where : > + cond1 - does not use min_max_column at all. > + cond2 - is an AND/OR tree with leaves in form > + "$FUNC$(min_max_column[, const])". > > RETURN > TRUE if cond passes the test > @@ -12061,7 +12072,8 @@ get_best_group_min_max(PARAM *param, SEL > > static bool > check_group_min_max_predicates(Item *cond, Item_field *min_max_arg_item, > - Field::imagetype image_type) > + Field::imagetype image_type, > + bool *has_min_max_arg, bool *has_other_arg) > { > DBUG_ENTER("check_group_min_max_predicates"); > DBUG_ASSERT(cond && min_max_arg_item); > @@ -12073,12 +12085,24 @@ check_group_min_max_predicates(Item *con > DBUG_PRINT("info", ("Analyzing: %s", ((Item_func*) cond)->func_name())); > List_iterator_fast<Item> li(*((Item_cond*) cond)->argument_list()); > Item *and_or_arg; > + Item_func::Functype func_type= ((Item_cond*) cond)->functype(); > + bool has_min_max= false, has_other= false; > while ((and_or_arg= li++)) > { > - if (!check_group_min_max_predicates(and_or_arg, min_max_arg_item, > - image_type)) > + /* > + The WHERE clause doesn't pass the condition if: > + (1) any subtree doesn't pass the condition or > + (2) the subtree passes the test, but it is an OR and it references both > + the min/max argument and other columns. > + */ > + if (!check_group_min_max_predicates(and_or_arg, min_max_arg_item, //1 > + image_type, > + &has_min_max, &has_other) || > + (func_type == Item_func::COND_OR_FUNC && has_min_max && has_other))//2 > DBUG_RETURN(FALSE); > } > + *has_min_max_arg= has_min_max || *has_min_max_arg; > + *has_other_arg= has_other || *has_other_arg; > DBUG_RETURN(TRUE); > } > > @@ -12112,6 +12136,10 @@ check_group_min_max_predicates(Item *con > if (cond_type == Item::FIELD_ITEM) > { > DBUG_PRINT("info", ("Analyzing: %s", cond->full_name())); > + if (min_max_arg_item->eq((Item_field*)cond, 1)) > + *has_min_max_arg= true; > + else > + *has_other_arg= true; > DBUG_RETURN(TRUE); > } > > @@ -12120,9 +12148,33 @@ check_group_min_max_predicates(Item *con > > /* Test if cond references only group-by or non-group fields. */ > Item_func *pred= (Item_func*) cond; > + Item_func::Functype pred_type= pred->functype(); > + DBUG_PRINT("info", ("Analyzing: %s", pred->func_name())); > + if (pred_type == Item_func::MULT_EQUAL_FUNC) > + { > + /* > + Check that each field in a multiple equality is either a constant or > + it is a reference to the min/max argument, or it doesn't contain the > + min/max argument at all. > + */ > + Item_equal_fields_iterator eq_it(*((Item_equal*)pred)); > + Item *eq_item; > + bool has_min_max= false, has_other= false; > + while ((eq_item= eq_it++)) > + { > + if (min_max_arg_item->eq(eq_item->real_item(), 1)) > + has_min_max= true; > + else > + has_other= true; > + } > + *has_min_max_arg= has_min_max || *has_min_max_arg; > + *has_other_arg= has_other || *has_other_arg; > + DBUG_RETURN(!(has_min_max && has_other)); > + } > + > Item **arguments= pred->arguments(); > Item *cur_arg; > - DBUG_PRINT("info", ("Analyzing: %s", pred->func_name())); > + bool has_min_max= false, has_other= false; > for (uint arg_idx= 0; arg_idx < pred->argument_count (); arg_idx++) > { > cur_arg= arguments[arg_idx]->real_item(); > @@ -12131,11 +12183,11 @@ check_group_min_max_predicates(Item *con > { > if (min_max_arg_item->eq(cur_arg, 1)) > { > - /* > - If pred references the MIN/MAX argument, check whether pred is a range > - condition that compares the MIN/MAX argument with a constant. > - */ > - Item_func::Functype pred_type= pred->functype(); > + has_min_max= true; > + /* > + If pred references the MIN/MAX argument, check whether pred is a range > + condition that compares the MIN/MAX argument with a constant. > + */ > if (pred_type != Item_func::EQUAL_FUNC && > pred_type != Item_func::LT_FUNC && > pred_type != Item_func::LE_FUNC && > @@ -12175,11 +12227,13 @@ check_group_min_max_predicates(Item *con > min_max_arg_item->field->cmp_type() != args[1]->result_type()))) > DBUG_RETURN(FALSE); > } > + else > + has_other= true; > } > else if (cur_arg->type() == Item::FUNC_ITEM) > { > - if (!check_group_min_max_predicates(cur_arg, min_max_arg_item, > - image_type)) > + if (!check_group_min_max_predicates(cur_arg, min_max_arg_item, image_type, > + &has_min_max, &has_other)) > DBUG_RETURN(FALSE); > } > else if (cur_arg->const_item()) > @@ -12192,7 +12246,11 @@ check_group_min_max_predicates(Item *con > } > else > DBUG_RETURN(FALSE); > + if(has_min_max && has_other) > + DBUG_RETURN(FALSE); > } > + *has_min_max_arg= has_min_max || *has_min_max_arg; > + *has_other_arg= has_other || *has_other_arg; > > DBUG_RETURN(TRUE); > } > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] Igor please review: EXPLAIN for DELETE
by Sergei Petrunia 07 Feb '13

07 Feb '13

Hello Igor, Hello Igor, Could you please review the following two patches: http://lists.askmonty.org/pipermail/commits/2012-December/003944.html http://lists.askmonty.org/pipermail/commits/2012-December/003945.html I think they implement the ideas about MDEV-3798 we've discussed earlier. BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

2 1

Re: [Maria-developers] mariadb-server not installable on Debian Testing AMD64
by Kristian Nielsen 06 Feb '13

06 Feb '13

Thomas Hackert <thackert(a)nexgo.de> writes: > I am not sure, how (and if) I can upload to f.a.o., so I will attach it > to this mail. Sorry for the inconvenience ... :( No inconvenience at all, mail is perfect. And indeed, the strace was illuminating. I think the issue is right here: [pid 32673] bind(14, {sa_family=AF_INET, sin_port=htons(3306), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EADDRNOTAVAIL (Cannot assign requested address) The server fails to start, because it cannot setup a socket to listen on the loopback interface on 127.0.0.1 The manpage for bind(2) says: EADDRNOTAVAIL A nonexistent interface was requested or the requested address was not local. I get the same error on my machine if I try to bind a socket to a non-existing interface. You can try this netcat command to do the same bind, it will probably fail in the same way: nc -l -s 127.0.0.1 -p 3306 So the problem most likely is that your system has a non-standard networking setup, with the normal loopback address 127.0.0.1 not configured. The default configuration file has this line: bind-address = 127.0.0.1 This will fail if the address 127.0.0.1 is not available on the system (normally it is). So I think you can edit /etc/mysql/my.cnf and change this line to use a valid IP address, or replace it with skip-networking to not use any IP address. Then try starting the server and see if it works. If it does, `apt-get install -f` might fix the installation. Can you confirm that your system has no 127.0.0.1 address? (It is somewhat annoying that the install fails in this case, but it does seem a rather rare setup). The second question is why you did not get any error log output with messages about this problem. I suppose this might be because you had no syslog installed. Elena gave you suggestions for how to resolve/investigate this further. But let me know if you have more problems with this. Thanks for your persistence in helping me get to the bottom of this! - Kristian.

2 6