- developers - lists.mariadb.org

[Maria-developers] GSOC 2020 - MDEV-11263 Aggregate Window Functions
by Tavneet Sarna 30 Mar '20

30 Mar '20

Hi everyone, As part of GSOC 2020, this is one of the two projects I am interested in pursuing. As guided by last years' comments by Varun and Vicențiu, I have set up a debugger and corresponding breakpoints in the Item_sum_sum::add and Item_sum_sp::add for a custom sum aggregate function to understand the code flow. I had a couple of queries regarding the same: 1. In *do_add *from decimal.c, there are three parts with comments - /* part 1 - MY_MAX(frac) ... min (frac) */, /* part 2 - MY_MIN(frac) ... MY_MIN(intg) */. Can someone please elaborate on what do the comments mean ? 2. In *Item_sum_sum::add_helper*, there is an unlikely branch for variable direct_added. Can someone please give an idea about when will direct added be true ? In fact in all the uses for direct_added, it is always in an unlikely branch in Item_sum.cc. Regards Tavneet

2 1

[Maria-developers] Guidance regarding GSoC Projects.
by Arhant Jain 27 Mar '20

27 Mar '20

Hello Everyone, I'm interested in the work associated with these projects. I tried to explore them on my level. It will be very helpful if the concerned developers can direct some more information regarding these projects. 1. Switching off/on separate tests in mysql-test-run suite 2. Add a bitmap for NULL/empty data 3. Speed up RowGroup data access methods reducing level of indirection Thanks Arhant

1 0

[Maria-developers] Regarding GSoc :ADD FULL OUTER JOIN
by Jeeukrishnan Kayshyap. 26 Mar '20

26 Mar '20

Hi ,I am a GSoC aspirant . I am trying to solve https://jira.mariadb.org/browse/MDEV-13648 . Firstly , I discussed a new algorithm with Igor Babev sir , but it was wrong as it will create space problem. Now , I am planning to add some code in Sql_yacc.yy : /*full join*/ after the right join where /*full join */ will contain : 1.left join implementation (similar to the implementation of left join in sql_yacc.yy (line no. 11956-11950)) The result table will be in a temporary table ->tab1. 2.Use of right join doing the swapping of table (similar to right join in this part ;(line no. 11992-12030)) the result of this will be stored in another temporary table ->tab2 3.Union all clause: implementation of union all clause to combine tab1 and tab2. So , Oleksandr Byelkin sir replied to me that , addition of UNION will make it more complicated.it could be done with less effort than adding full UNION. But at present I am not able to get any idea how to reduce the complexity or any kind of new idea to implement it . Please guide me in the right direction and help me to solve this issue.

2 4

Re: [Maria-developers] fc90a39426c: MDEV-20604: Duplicate key value is silently truncated to 64 characters in print_keydup_error
by Sergei Golubchik 25 Mar '20

25 Mar '20

Hi, Oleksandr! Looks good. Thanks for tests. Ok to push. One comment, see below. On Mar 25, Oleksandr Byelkin wrote: > revision-id: fc90a39426c (mariadb-10.2.31-87-gfc90a39426c) > parent(s): fb74de97287 > author: Oleksandr Byelkin <sanja(a)mariadb.com> > committer: Oleksandr Byelkin <sanja(a)mariadb.com> > timestamp: 2020-03-24 10:34:14 +0100 > message: > > MDEV-20604: Duplicate key value is silently truncated to 64 characters in print_keydup_error > > Added indication of truncated string for "s" and "M" formats > > diff --git a/unittest/mysys/my_vsnprintf-t.c b/unittest/mysys/my_vsnprintf-t.c > index 6ba0a42cf7e..872e88ddd7e 100644 > --- a/unittest/mysys/my_vsnprintf-t.c > +++ b/unittest/mysys/my_vsnprintf-t.c > @@ -168,14 +184,14 @@ int main(void) > test1("M with positional: 0 \"Internal error/check (Not system error)\"", > "M with positional: %1$M", 0); > > - test1("M with width: 0 \"Internal error/ch", > + test1("M with width: 0 \"Internal error...", > "M with width: %.20M", 0); > - test1("M with width positional: 0 \"Internal error/ch", > + test1("M with width positional: 0 \"Internal error...", > "M with width positional: %2$.*1$M", 20, 0); > > - test_w_len("M small buf: 0 \"In", > + test_w_len("M small buf: 0 \"..", > 19, "M small buf: %M", 0); > - test_w_len("M small buf positional: 0 \"In", > + test_w_len("M small buf positional: 0 \"..", > 30, "M small buf positional: %1$M", 0); > > return exit_status(); We didn't discuss truncation when the buffer is too small, only truncation by width. It's not at all clear why truncation by small buffer should also use ellipsis. But it's not clear that it shouldn't either. I don't have a strong opinion in favor of either, so feel free to keep it your way. Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

1 0

[Maria-developers] GSOC 2020 - MDEV-383
by Tavneet Sarna 25 Mar '20

25 Mar '20

Hi, I was looking into MDEV-383 <https://jira.mariadb.org/browse/MDEV-383> - "*Evaluate subquery predicates earlier or later depending on their SELECTIVITY*" as a possible GSOC 2020 project and I have a few queries about the same. 1. "There are a lot of subquery conditions out there that are inexpensive to evaluate and have good selectivity." - How is the cost of a subquery predicate calculated ? 2. One of the assumptions in MDEV-83 <https://jira.mariadb.org/browse/MDEV-83> was that "selectivity of a subquery predicate cannot be estimated during optimization." - what has changed since that we are looking into estimating selectivity ( or maybe my understanding of the task is incorrect so I would be grateful if you could just point me to any resources regarding this). 3. Igor had responded on the jira - "for starters, you could introduce an option that would allow not to push correlated subquery predicates to joined tables." - did you mean a command line argument/option and I don't get how this would help with the main task. 4. Also, can anyone please point me to the classes where predicate cost and join plans are evaluated ? Apologies if the queries are a bit rudimentary. Regards Tavneet

1 0

[Maria-developers] Regarding GSoc :ADD FULL OUTER JOIN
by Jeeukrishnan Kayshyap. 24 Mar '20

24 Mar '20

HI , to implement FULL OUTER JOIN between T1 and T2 can we do "T1 Left Outer Join T2 union all T2 Anti Join T1" ?? Is this method already tried ? Please guide me to solve this issue and if possible someone please reply to my last email.

1 0

[Maria-developers] MDEV-14296: Moving wsrep_sst_common to /usr/libexec
by Vicențiu Ciorbaru 23 Mar '20

23 Mar '20

Hi Otto! I am doing a push towards cleaning up as much as possible from our packaging issues. Starting with the older ones: *Background:* wsrep_sst_common.sh is a script that should never be run standalone, it is rather a helper script used for other wsrep_* scripts. As such, it *does not really belong in /usr/bin*. See MDEV-14296 [1] for more details. <https://jira.mariadb.org/browse/MDEV-14296> It looks like rpm packagers - read Fedora - [1a] require us to ship wsrep_sst_common in */usr/libexec* (or alternatively /usr/share, but /usr/libexec makes much more sense semantically to me, after reading the FHS [2]). There is PR #603 [3] from Daniel Black, which has some missing changes, which I've done myself, namely updating the debian/*.install files to put the file in the correct location and a change in path for our apparmor profile [4] [5]. *My question for you is*: Is this acceptable to do in 10.1 (merge PR 603 + commits [4] [5] into 10.1) from a packaging point of view? The debian packaging policy here [6], allows one to follow FSH version 3.0 and does not mention any exception for libexec. <https://www.debian.org/doc/debian-policy/ch-opersys.html> I have tested this and I have not encountered any regressions while upgrading and the script seems to function as intended (although I am not an expert DBA to make use of wsrep_* scripts). Thanks, Vicențiu [1] https://jira.mariadb.org/browse/MDEV-14296 [1a] https://src.fedoraproject.org/rpms/mariadb/blob/ecb40d449c382ea7e4052c75d5d… [2] https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch04s07.html [3] https://github.com/MariaDB/server/pull/603 [4] https://github.com/MariaDB/server/commit/8042265659656c312f39304a5dc60de708… [5] https://github.com/MariaDB/server/commit/dbed51d86e4aae9b8a54d6b5aa31c974fe… [6] https://www.debian.org/doc/debian-policy/ch-opersys.html

1 0

[Maria-developers] Regarding GSoc :ADD FULL OUTER JOIN
by Jeeukrishnan Kayshyap. 23 Mar '20

23 Mar '20

Hi , I studied the join_table part of sql_yacc.yy and the the convert_right_join() part from sql_parse.cc . Sir to add the full outer join : if we add a /*Full outer join variant */ , where first we will do left outer join and then right outer join in that , after that we can use the "UNION" part in /*Full outer join variant */ to join the left outer join and right outer join table to get the full outer join table. Will this approach gonna work Sir? please guide me for this issue.Is it completely wrong or need some modification?

2 1

[Maria-developers] Regarding GSoC : ADD FULL OUTER JOIN
by Jeeukrishnan Kayshyap. 23 Mar '20

23 Mar '20

What are the other thing we need to consider if we include *st_select_lex::convert_full_join() similar to *st_select_lex::convert_right_join() ?

1 0

Re: [Maria-developers] 4b142536a24: MDEV-20604: Duplicate key value is silently truncated to 64 characters in print_keydup_error
by Sergei Golubchik 22 Mar '20

22 Mar '20

Hi, Oleksandr! On Mar 22, Oleksandr Byelkin wrote: > revision-id: 4b142536a24 (mariadb-10.2.31-56-g4b142536a24) > parent(s): ed21202a14e > author: Oleksandr Byelkin <sanja(a)mariadb.com> > committer: Oleksandr Byelkin <sanja(a)mariadb.com> > timestamp: 2020-03-18 08:25:13 +0100 > message: > > MDEV-20604: Duplicate key value is silently truncated to 64 characters in print_keydup_error > > Added indication of truncated string with special width/precission suffix ':'. The comment looks wrong :) > diff --git a/client/mysql_upgrade.c b/client/mysql_upgrade.c > index 0bbb8a149ff..003701514cf 100644 > --- a/client/mysql_upgrade.c > +++ b/client/mysql_upgrade.c > @@ -501,10 +499,10 @@ static void find_tool(char *tool_executable_name, const char *tool_name, > last_fn_libchar -= 6; > } > > - len= (int)(last_fn_libchar - self_name); > - > - my_snprintf(tool_executable_name, FN_REFLEN, "%.*s%c%s", > - len, self_name, FN_LIBCHAR, tool_name); > + last_fn_libchar[0]= 0; > + my_snprintf(tool_executable_name, FN_REFLEN, "%s%c%s", > + self_name, FN_LIBCHAR, tool_name); > + last_fn_libchar[0]= FN_LIBCHAR; Okay, but why not to use %b here? > } > > if (opt_verbose) > diff --git a/client/mysqltest.cc b/client/mysqltest.cc > index 60a203ccedd..045ab566fdb 100644 > --- a/client/mysqltest.cc > +++ b/client/mysqltest.cc > @@ -9534,6 +9533,7 @@ int main(int argc, char **argv) > case Q_LET: do_let(command); break; > case Q_EVAL_RESULT: > die("'eval_result' command is deprecated"); > + break; // never called but keep compiler calm wouldn't it be better to specify __attribute__ ((noreturn)) for die()? > case Q_EVAL: > case Q_EVALP: > case Q_QUERY_VERTICAL: > diff --git a/strings/my_vsnprintf.c b/strings/my_vsnprintf.c > index ad3517e4252..9ae10f337cc 100644 > --- a/strings/my_vsnprintf.c > +++ b/strings/my_vsnprintf.c > @@ -28,7 +28,9 @@ > #define LENGTH_ARG 1 > #define WIDTH_ARG 2 > #define PREZERO_ARG 4 > -#define ESCAPED_ARG 8 > +#define ESCAPED_ARG 8 > +// Show truncation of string argument > +#define SH_TRUNC_ARG 16 you don't really need a flag, because process_str_arg should always do it. By the way, in your code it doesn't. It doesn't print dots for %M and for %1$s. > > typedef struct pos_arg_info ARGS_INFO; > typedef struct print_info PRINT_INFO; > @@ -198,18 +199,42 @@ static char *process_str_arg(CHARSET_INFO *cs, char *to, const char *end, > size_t width, char *par, uint print_type) > { > int well_formed_error; > - size_t plen, left_len= (size_t) (end - to) + 1; > + uint dots= 0; > + size_t plen, left_len= (size_t) (end - to) + 1, slen=0; > if (!par) > par = (char*) "(null)"; > > - plen= strnlen(par, width); > + plen= slen= strnlen(par, width + 1); > + if (plen > width) > + plen= width; > if (left_len <= plen) > plen = left_len - 1; > + if ((slen > plen) && (print_type & SH_TRUNC_ARG)) > + { > + if (plen < 3) > + { > + dots= plen; > + plen= 0; > + } > + else > + { > + plen-= 3; > + dots= 3; > + } > + } > + > plen= my_well_formed_length(cs, par, par + plen, width, &well_formed_error); > if (print_type & ESCAPED_ARG) > to= backtick_string(cs, to, end, par, plen, '`'); > else > to= strnmov(to,par,plen); > + > + if (dots) > + { > + for (; dots; dots--) > + *(to++)= '.'; > + *(to)= 0; > + } This should look strange when truncating a quoted string. See the test case below > return to; > } > > diff --git a/unittest/mysys/my_vsnprintf-t.c b/unittest/mysys/my_vsnprintf-t.c > index 6ba0a42cf7e..e8831f7c1f4 100644 > --- a/unittest/mysys/my_vsnprintf-t.c > +++ b/unittest/mysys/my_vsnprintf-t.c > @@ -99,12 +99,12 @@ int main(void) > test1("Width is ignored for strings <x> <y>", > "Width is ignored for strings <%04s> <%5s>", "x", "y"); > > - test1("Precision works for strings <abcde>", > + test1("Precision works for strings <ab...>", > "Precision works for strings <%.5s>", "abcdef!"); > > - test1("Flag '`' (backtick) works: `abcd` `op``q` (mysql extension)", This > - "Flag '`' (backtick) works: %`s %`.4s (mysql extension)", This > - "abcd", "op`qrst"); > + test1("Flag '`' (backtick) works: `abcd` `op``q`... (mysql extension)", > + "Flag '`' (backtick) works: %`s %`.7s (mysql extension)", > + "abcd", "op`qrstuuuuuuuuu"); Here, the test case with a quoted string. First, you print 10 characters instead of 7. Second, you put dots after the closing backtick, which looks confusing, the truncation happens before the backtick. Third, as you can see, you cannot use strlen() for a string length because there can be backticks inside the string. perhaps it would be simpler to move ... inside backtick_string() > > test1("Length modifiers work: 1 * -1 * 2 * 3", > "Length modifiers work: %d * %ld * %lld * %zd", 1, -1L, 2LL, (size_t)3); > @@ -125,7 +125,7 @@ int main(void) > test1("Asterisk '*' as a width works: < 4>", > "Asterisk '*' as a width works: <%*d>", 5, 4); > > - test1("Asterisk '*' as a precision works: <qwerty>", > + test1("Asterisk '*' as a precision works: <qwe...>", > "Asterisk '*' as a precision works: <%.*s>", 6, "qwertyuiop"); please add tests for ...truncation in %M and %1$s > > test1("Positional arguments for a width: < 4>", > Regards, Sergei VP of MariaDB Server Engineering and security(a)mariadb.org

1 0