- developers - lists.mariadb.org

Re: [Maria-developers] db9822d: MDEV-8931: (server part of) session state tracking
by Sergei Golubchik 31 Aug '16

31 Aug '16

Hi, Oleksandr! Looks ok to push, thanks! On Aug 09, Oleksandr Byelkin wrote: > revision-id: db9822df31917c3d0162322a77bcd4433bfb705e (mariadb-10.2.1-15-gdb9822d) > parent(s): b5e0f70e04cec625c99760e5bfb28bb31bc972ed > committer: Oleksandr Byelkin > timestamp: 2016-08-09 15:49:30 +0200 > message: > > MDEV-8931: (server part of) session state tracking > > Postreview fixes. > New MySQL tests fixes. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] 4949925: MDEV-10421 duplicate CHECK CONSTRAINTs.
by Sergei Golubchik 31 Aug '16

31 Aug '16

Hi, Alexey! Looks good, thanks! See few comments below, then ok to push! On Aug 30, Alexey Botchkov wrote: > revision-id: 49499250c04b631bd908945b7c59d6affaeb2703 (mariadb-10.2.1-9-g4949925) > parent(s): 08683a726773f8cdf16a4a3dfb3920e5f7842481 > committer: Alexey Botchkov > timestamp: 2016-08-30 16:43:05 +0400 > message: > > MDEV-10421 duplicate CHECK CONSTRAINTs. > > mysql_prepare_create_table fixed so it doesn't let duplicating > constraint names. Syntax for CONSTRAINT IF NOT EXISTS added > and handled in mysql_alter_table. > > diff --git a/sql/field.h b/sql/field.h > index 13a0e91..7b271b0 100644 > --- a/sql/field.h > +++ b/sql/field.h > @@ -606,6 +606,7 @@ class Virtual_column_info: public Sql_alloc > /* Flag indicating that the field is physically stored in the database */ > bool stored_in_db; > bool utf8; /* Already in utf8 */ > + bool create_if_not_exists; You apparently forgot to remove create_if_not_exists. It doesn't seem to be used anywhere. > /* The expression to compute the value of the virtual column */ > Item *expr_item; > /* Text representation of the defining expression */ > diff --git a/sql/sql_table.cc b/sql/sql_table.cc > index e745fe8..c120fbc 100644 > --- a/sql/sql_table.cc > +++ b/sql/sql_table.cc > @@ -6130,6 +6146,38 @@ handle_if_exists_options(THD *thd, TABLE *table, Alter_info *alter_info) > } > #endif /*WITH_PARTITION_STORAGE_ENGINE*/ > > + /* ADD CONSTRAINT IF NOT EXISTS. */ > + { > + List_iterator<Virtual_column_info> it(alter_info->check_constraint_list); > + Virtual_column_info *check; > + TABLE_SHARE *share= table->s; > + uint c; > + while ((check=it++)) > + { > + if (!check->flags /* & Alter_info::CHECK_CONSTRAINT_IF_NOT_EXISTS */ && why is that? because CHECK_CONSTRAINT_IF_NOT_EXISTS is the only possible flag now? I'd still prefer it a bit more future-proof, better to check the flag here. > + check->name.length) > + continue; > + check->flags= 0; > + for (c= share->field_check_constraints; > + c < share->table_check_constraints ; c++) > + { > + Virtual_column_info *dup= table->check_constraints[c]; > + if (dup->name.length == check->name.length && > + my_strcasecmp(system_charset_info, > + check->name.str, dup->name.str) == 0) > + { > + push_warning_printf(thd, Sql_condition::WARN_LEVEL_NOTE, > + ER_DUP_CONSTRAINT_NAME, ER_THD(thd, ER_DUP_CONSTRAINT_NAME), > + "CHECK", check->name.str); > + it.remove(); > + if (alter_info->check_constraint_list.elements == 0) > + alter_info->flags&= ~Alter_info::ALTER_ADD_CHECK_CONSTRAINT; > + > + break; > + } > + } > + } > + } > DBUG_VOID_RETURN; > } You seem to forgot to add #define ER_FK_DUP_NAME ER_DUP_CONSTRAINT_NAME to mysql.h. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MDEV-10702 Crash in SET STATEMENT FOR EXECUTE
by Alexander Barkov 31 Aug '16

31 Aug '16

Hello Sergei, Please review a patch for mdev-10702.diff Thanks!

1 0

Re: [Maria-developers] [Maria-discuss] Known limitation with TokuDB in Read Free Replication & parallel replication ?
by Kristian Nielsen 30 Aug '16

30 Aug '16

Rich Prohaska <prohaska7(a)gmail.com> writes: > I rearranged the tokudb lock request wait function a little bit, and got > the lock tree unit tests to compile again (since the API changed). commit > on this tree: https://github.com/prohaska7/mariadb-server/tree/toku_opr3 Thanks! So I took a look at the complete patch we have so far, thinking what remains to do to get this included in main MariaDB. Do you have any suggestions for how/whether to upstream the TokuDB parts? I assume the new kill_query functionality would make sense to upstream. The callback to report waits could be upstreamed, but would not be used in non-mariadb builds, probably. And the actual tokudb_lock_wait_needed_callback() would probably not be upstreamed (or would go in #ifdef MARIADB_BASE_VERSION perhaps). Any thoughts? I still need to look into the should_retry_lock_requests, currently I get hangs if it is enabled. And I want to see if the wait_needed_callback interface can be cleaned up a bit. Otherwise the patch looks fairly complete to me, do you agree? I think this could go in 10.1, to fix optimistic parallel replication with TokuDB (which does not currently work at all). - Kristian.

2 1

Re: [Maria-developers] [Commits] 2a4ed5f: MDEV-10017: Get unexpected `Empty Set` for correlated subquery with aggregate functions (part 1)
by Sergey Petrunia 29 Aug '16

29 Aug '16

Hi Sanja, Table t2 is the testcase is redundant. Ok to push after this is addressed. On Mon, Aug 29, 2016 at 01:50:09PM +0200, Oleksandr Byelkin wrote: > revision-id: 2a4ed5f7a74860608cb36b761eb139a57cbd4463 (mariadb-10.1.16-15-g2a4ed5f) > parent(s): 76bd9f655a42dc1e21acfa26650a0cb76c70a22e > committer: Oleksandr Byelkin > timestamp: 2016-08-29 13:50:09 +0200 > message: > > MDEV-10017: Get unexpected `Empty Set` for correlated subquery with aggregate functions (part 1) > > Make aggregate function dependency visible. > > --- > mysql-test/r/func_group.result | 19 +++++++++++++++++++ > mysql-test/r/subselect3.result | 1 + > mysql-test/r/subselect3_jcl6.result | 1 + > mysql-test/r/subselect_mat.result | 1 + > mysql-test/r/subselect_sj_mat.result | 1 + > mysql-test/t/func_group.test | 11 +++++++++++ > sql/item_sum.cc | 10 ++++++++++ > sql/share/errmsg-utf8.txt | 3 +++ > 8 files changed, 47 insertions(+) > > diff --git a/mysql-test/r/func_group.result b/mysql-test/r/func_group.result > index 0d9d4ff..4bc5ad1 100644 > --- a/mysql-test/r/func_group.result > +++ b/mysql-test/r/func_group.result > @@ -2340,5 +2340,24 @@ companynr AVG(fld1) avg1 avg2 > 37 9223372036854775805.0000 9223372036854775805 9223372036854775805 > DROP TABLE t1; > # > +# case where aggregate resolved in the local SELECT > +# but outer ones are checked > +# > +create table t10 (a int , b int, c int); > +insert into t10 values (0,0,0),(1,1,1); > +create table t11 as select * from t10; > +create table t12 as select * from t10; > +create table t2 as select * from t10; > +explain extended select a from t10 where c<3 or a in (select c from t12 union select max(t10.b) from t11 group by t11.c); > +id select_type table type possible_keys key key_len ref rows filtered Extra > +1 PRIMARY t10 ALL NULL NULL NULL NULL 2 100.00 Using where > +2 DEPENDENT SUBQUERY t12 ALL NULL NULL NULL NULL 2 100.00 Using where > +3 DEPENDENT UNION t11 ALL NULL NULL NULL NULL 2 100.00 Using temporary > +NULL UNION RESULT <union2,3> ALL NULL NULL NULL NULL NULL NULL > +Warnings: > +Note 1276 Field or reference 'test.t10.b' of SELECT #3 was resolved in SELECT #1 > +Note 1003 select `test`.`t10`.`a` AS `a` from `test`.`t10` where ((`test`.`t10`.`c` < 3) or <expr_cache><`test`.`t10`.`a`,`test`.`t10`.`b`>(<in_optimizer>(`test`.`t10`.`a`,<exists>(select `test`.`t12`.`c` from `test`.`t12` where (<cache>(`test`.`t10`.`a`) = `test`.`t12`.`c`) union select max(`test`.`t10`.`b`) from `test`.`t11` group by `test`.`t11`.`c` having (<cache>(`test`.`t10`.`a`) = <ref_null_helper>(max(`test`.`t10`.`b`))))))) > +drop table t10,t11,t12,t2; > +# > # End of 10.1 tests > # > diff --git a/mysql-test/r/subselect3.result b/mysql-test/r/subselect3.result > index 24d9f0d..052b46d 100644 > --- a/mysql-test/r/subselect3.result > +++ b/mysql-test/r/subselect3.result > @@ -889,6 +889,7 @@ ERROR 42S22: Unknown column 'c' in 'field list' > SHOW WARNINGS; > Level Code Message > Note 1276 Field or reference 'test.t1.a' of SELECT #3 was resolved in SELECT #2 > +Note 1981 Aggregate function 'count()' of SELECT #3 belongs to SELECT #2 > Note 1276 Field or reference 'test.t1.c' of SELECT #3 was resolved in SELECT #2 > Error 1054 Unknown column 'c' in 'field list' > DROP TABLE t1; > diff --git a/mysql-test/r/subselect3_jcl6.result b/mysql-test/r/subselect3_jcl6.result > index 19d3d25..28f4e40 100644 > --- a/mysql-test/r/subselect3_jcl6.result > +++ b/mysql-test/r/subselect3_jcl6.result > @@ -899,6 +899,7 @@ ERROR 42S22: Unknown column 'c' in 'field list' > SHOW WARNINGS; > Level Code Message > Note 1276 Field or reference 'test.t1.a' of SELECT #3 was resolved in SELECT #2 > +Note 1981 Aggregate function 'count()' of SELECT #3 belongs to SELECT #2 > Note 1276 Field or reference 'test.t1.c' of SELECT #3 was resolved in SELECT #2 > Error 1054 Unknown column 'c' in 'field list' > DROP TABLE t1; > diff --git a/mysql-test/r/subselect_mat.result b/mysql-test/r/subselect_mat.result > index d0c5c2e..bfb40b9 100644 > --- a/mysql-test/r/subselect_mat.result > +++ b/mysql-test/r/subselect_mat.result > @@ -1139,6 +1139,7 @@ id select_type table type possible_keys key key_len ref rows filtered Extra > 3 DEPENDENT SUBQUERY t3 ALL NULL NULL NULL NULL 4 100.00 Using where > Warnings: > Note 1276 Field or reference 'test.t1.b' of SELECT #3 was resolved in SELECT #1 > +Note 1981 Aggregate function 'max()' of SELECT #3 belongs to SELECT #1 > Note 1003 select `test`.`t1`.`a` AS `a` from `test`.`t1` group by `test`.`t1`.`a` having <expr_cache><`test`.`t1`.`a`,`test`.`t1`.`b`,max(`test`.`t1`.`b`),max(`test`.`t1`.`b`)>(<in_optimizer>(`test`.`t1`.`a`,<exists>(select `test`.`t2`.`c` from `test`.`t2` where (<nop>(<expr_cache><`test`.`t2`.`d`,`test`.`t1`.`b`,max(`test`.`t1`.`b`),max(`test`.`t1`.`b`)>(<in_optimizer>(`test`.`t2`.`d`,<exists>(select `test`.`t3`.`e` from `test`.`t3` where (max(`test`.`t1`.`b`) = `test`.`t3`.`e`) having (<cache>(`test`.`t2`.`d`) >= <ref_null_helper>(`test`.`t3`.`e`)))))) and (<cache>(`test`.`t1`.`a`) = `test`.`t2`.`c`))))) > select a from t1 group by a > having a in (select c from t2 where d >= some(select e from t3 where max(b)=e)); > diff --git a/mysql-test/r/subselect_sj_mat.result b/mysql-test/r/subselect_sj_mat.result > index e846549..366a03b 100644 > --- a/mysql-test/r/subselect_sj_mat.result > +++ b/mysql-test/r/subselect_sj_mat.result > @@ -1173,6 +1173,7 @@ id select_type table type possible_keys key key_len ref rows filtered Extra > 3 DEPENDENT SUBQUERY t3 ALL NULL NULL NULL NULL 4 100.00 Using where > Warnings: > Note 1276 Field or reference 'test.t1.b' of SELECT #3 was resolved in SELECT #1 > +Note 1981 Aggregate function 'max()' of SELECT #3 belongs to SELECT #1 > Note 1003 select `test`.`t1`.`a` AS `a` from `test`.`t1` group by `test`.`t1`.`a` having <expr_cache><`test`.`t1`.`a`,`test`.`t1`.`b`,max(`test`.`t1`.`b`),max(`test`.`t1`.`b`)>(<in_optimizer>(`test`.`t1`.`a`,<exists>(select `test`.`t2`.`c` from `test`.`t2` where (<nop>(<expr_cache><`test`.`t2`.`d`,`test`.`t1`.`b`,max(`test`.`t1`.`b`),max(`test`.`t1`.`b`)>(<in_optimizer>(`test`.`t2`.`d`,<exists>(select `test`.`t3`.`e` from `test`.`t3` where (max(`test`.`t1`.`b`) = `test`.`t3`.`e`) having (<cache>(`test`.`t2`.`d`) >= <ref_null_helper>(`test`.`t3`.`e`)))))) and (<cache>(`test`.`t1`.`a`) = `test`.`t2`.`c`))))) > select a from t1 group by a > having a in (select c from t2 where d >= some(select e from t3 where max(b)=e)); > diff --git a/mysql-test/t/func_group.test b/mysql-test/t/func_group.test > index 47477c7..4a8b76b 100644 > --- a/mysql-test/t/func_group.test > +++ b/mysql-test/t/func_group.test > @@ -1617,5 +1617,16 @@ SELECT companynr, AVG(fld1), AVG(fld1)<<0 AS avg1, CAST(AVG(fld1) AS UNSIGNED)<< > DROP TABLE t1; > > --echo # > +--echo # case where aggregate resolved in the local SELECT > +--echo # but outer ones are checked > +--echo # > +create table t10 (a int , b int, c int); > +insert into t10 values (0,0,0),(1,1,1); > +create table t11 as select * from t10; > +create table t12 as select * from t10; > +create table t2 as select * from t10; > +explain extended select a from t10 where c<3 or a in (select c from t12 union select max(t10.b) from t11 group by t11.c); > +drop table t10,t11,t12,t2; > +--echo # > --echo # End of 10.1 tests > --echo # > diff --git a/sql/item_sum.cc b/sql/item_sum.cc > index 1cfee1a..2a2b7f9 100644 > --- a/sql/item_sum.cc > +++ b/sql/item_sum.cc > @@ -380,6 +380,16 @@ bool Item_sum::register_sum_func(THD *thd, Item **ref) > sl->master_unit()->item->with_sum_func= 1; > } > thd->lex->current_select->mark_as_dependent(thd, aggr_sel, NULL); > + > + if ((thd->lex->describe & DESCRIBE_EXTENDED) && aggr_sel) > + { > + push_warning_printf(thd, Sql_condition::WARN_LEVEL_NOTE, > + ER_WARN_AGGFUNC_DEPENDENCE, > + ER_THD(thd, ER_WARN_AGGFUNC_DEPENDENCE), > + func_name(), > + thd->lex->current_select->select_number, > + aggr_sel->select_number); > + } > return FALSE; > } > > diff --git a/sql/share/errmsg-utf8.txt b/sql/share/errmsg-utf8.txt > index 709ac55..a6eaf2b 100644 > --- a/sql/share/errmsg-utf8.txt > +++ b/sql/share/errmsg-utf8.txt > @@ -7139,3 +7139,6 @@ ER_KILL_QUERY_DENIED_ERROR > ER_NO_EIS_FOR_FIELD > eng "Engine-independent statistics are not collected for column '%s'" > ukr "Незалежна від типу таблиці статистика не збирається для стовбця '%s'" > +ER_WARN_AGGFUNC_DEPENDENCE > + eng "Aggregate function '%-.192s)' of SELECT #%d belongs to SELECT #%d" > + ukr "Агрегатна функція '%-.192s)' з SELECTу #%d належить до SELECTу #%d" > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] c584f16: MDEV-10017: Get unexpected `Empty Set` for correlated subquery with aggregate functions (part 1)
by Sergey Petrunia 29 Aug '16

29 Aug '16

Hi Sanja, On Thu, Jun 23, 2016 at 05:50:08PM +0200, Oleksandr Byelkin wrote: > revision-id: c584f1655ef45dafd2b5d0eb55ca21421de9a266 (mariadb-10.1.14-25-gc584f16) > parent(s): 773ce408762b5f8256d4053b6d0d418d15657b92 > committer: Oleksandr Byelkin > timestamp: 2016-06-23 17:50:07 +0200 > message: > > MDEV-10017: Get unexpected `Empty Set` for correlated subquery with aggregate functions (part 1) > > Make aggregate function dependency visible. > * I was able to get a crash with the patch. The details are below. * The wording of the warning feels odd: > Field or reference 'test.t10.b' of SELECT #3 was resolved in SELECT #1 > ... > Aggregate function 'max()' of SELECT #3 belongs to SELECT #1 + I like how the wording of the new warning is similar to the old one. - It is not immediately clear what does "belong to" mean. I would change it to " Aggregate function 'max()' of SELECT #3 is aggregated in SELECT #1 " This looks much clearer. OTOH, "Aggregate .. is aggregated" doesn't sound very good. We probably need to consult a native speaker (of both English and SQL :). == Details about the crash == create table ten(a int); insert into ten values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); create table t10 (a int , b int, c int); insert into t10 select a,a,a from ten; create table t11 as select * from t10; create table t2 as select * from t10; explain extended select a from t10 where c<3 or a in (select c from t12 union select max(t10.b) from t11 group by t11.c); Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ffff7f1fb00 (LWP 4805)] 0x0000555555d0dd3f in Item_sum::register_sum_func (this=0x7fff89845680, thd=0x7fff9cbf6070, ref=0x7fff898457d0) at /home/psergey/dev-git/10.1/sql/item_sum.cc:391 (gdb) wher #0 0x0000555555d0dd3f in Item_sum::register_sum_func (this=0x7fff89845680, thd=0x7fff9cbf6070, ref=0x7fff898457d0) at /home/psergey/dev-git/10.1/sql/item_sum.cc:391 #1 0x0000555555d0d6f2 in Item_sum::check_sum_func (this=0x7fff89845680, thd=0x7fff9cbf6070, ref=0x7fff898457d0) at /home/psergey/dev-git/10.1/sql/item_sum.cc:177 #2 0x0000555555d1041e in Item_sum_hybrid::fix_fields (this=0x7fff89845680, thd=0x7fff9cbf6070, ref=0x7fff898457d0) at /home/psergey/dev-git/10.1/sql/item_sum.cc:1222 #3 0x00005555559c3cf2 in setup_fields (thd=0x7fff9cbf6070, ref_pointer_array=0x7fff898480f0, fields=..., mark_used_columns=MARK_COLUMNS_READ, sum_func_list=0x7fff89847ec8, allow_sum_func=true) at /home/psergey/dev-git/10.1/sql/sql_base.cc:7899 #4 0x0000555555a5b886 in JOIN::prepare (this=0x7fff89847b68, rref_pointer_array=0x7fff89845138, tables_init=0x7fff89845830, wild_num=0, conds_init=0x0, og_num=1, order_init=0x0, skip_order_by=true, group_init=0x7fff89845f38, having_init=0x0, proc_param_init=0x0, select_lex_arg=0x7fff89844ec0, unit_arg=0x7fff89844078) at /home/psergey/dev-git/10.1/sql/sql_select.cc:795 #5 0x0000555555af7082 in st_select_lex_unit::prepare (this=0x7fff89844078, thd_arg=0x7fff9cbf6070, sel_result=0x7fff89846118, additional_options=268435456) at /home/psergey/dev-git/10.1/sql/sql_union.cc:454 #6 0x0000555555d04d2c in subselect_union_engine::prepare (this=0x7fff89846140, thd_arg=0x7fff9cbf6070) at /home/psergey/dev-git/10.1/sql/item_subselect.cc:3590 #7 0x0000555555cfabe4 in Item_subselect::fix_fields (this=0x7fff89845f88, thd_param=0x7fff9cbf6070, ref=0x7fff89846270) at /home/psergey/dev-git/10.1/sql/item_subselect.cc:262 #8 0x0000555555d03f3d in Item_in_subselect::fix_fields (this=0x7fff89845f88, thd_arg=0x7fff9cbf6070, ref=0x7fff89846270) at /home/psergey/dev-git/10.1/sql/item_subselect.cc:3253 #9 0x0000555555c999cf in Item_cond::fix_fields (this=0x7fff89846178, thd=0x7fff9cbf6070, ref=0x7fff89846e98) at /home/psergey/dev-git/10.1/sql/item_cmpfunc.cc:4523 #10 0x00005555559c5bf1 in setup_conds (thd=0x7fff9cbf6070, tables=0x7fff89843330, leaves=..., conds=0x7fff89846e98) at /home/psergey/dev-git/10.1/sql/sql_base.cc:8627 #11 0x0000555555aa3eab in setup_without_group (thd=0x7fff9cbf6070, ref_pointer_array=0x7fff89846fd8, tables=0x7fff89843330, leaves=..., fields=..., all_fields=..., conds=0x7fff89846e98, order=0x0, group=0x0, hidden_group_fields=0x7fff89846d78, reserved=0x7fff9cbfa454) at /home/psergey/dev-git/10.1/sql/sql_select.cc:645 #12 0x0000555555a5b94b in JOIN::prepare (this=0x7fff89846a50, rref_pointer_array=0x7fff9cbfa430, tables_init=0x7fff89843330, wild_num=0, conds_init=0x7fff89846178, og_num=0, order_init=0x0, skip_order_by=false, group_init=0x0, having_init=0x0, proc_param_init=0x0, select_lex_arg=0x7fff9cbfa1b8, unit_arg=0x7fff9cbf9ab8) at /home/psergey/dev-git/10.1/sql/sql_select.cc:799 #13 0x0000555555a64f91 in mysql_select (thd=0x7fff9cbf6070, rref_pointer_array=0x7fff9cbfa430, tables=0x7fff89843330, wild_num=0, fields=..., conds=0x7fff89846178, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748612, result=0x7fff89846378, unit=0x7fff9cbf9ab8, select_lex=0x7fff9cbfa1b8) at /home/psergey/dev-git/10.1/sql/sql_select.cc:3415 #14 0x0000555555a9a6f5 in mysql_explain_union (thd=0x7fff9cbf6070, unit=0x7fff9cbf9ab8, result=0x7fff89846378) at /home/psergey/dev-git/10.1/sql/sql_select.cc:24737 #15 0x0000555555a2a9aa in execute_sqlcom_select (thd=0x7fff9cbf6070, all_tables=0x7fff89843330) at /home/psergey/dev-git/10.1/sql/sql_parse.cc:5834 #16 0x0000555555a20c72 in mysql_execute_command (thd=0x7fff9cbf6070) at /home/psergey/dev-git/10.1/sql/sql_parse.cc:2961 #17 0x0000555555a2e4be in mysql_parse (thd=0x7fff9cbf6070, rawbuf=0x7fff89843088 "explain extended select a from t10 where c<3 or a in (select c from t12 union select max(t10.b) from t11 group by t11.c)", length=120, parser_state=0x7ffff7f1e630) at /home/psergey/dev-git/10.1/sql/sql_parse.cc:7319 BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] Aggregate Functions
by Varun Gupta 26 Aug '16

26 Aug '16

Hi, As GSOC is nearing its end, I have written a blog which contains how can aggregate functions be written and executed to get the desired results.In the blog I also have added the technical details of this project on aggregate functions. Here are the links: Jira issue: https://jira.mariadb.org/browse/MDEV-7773 Blog Link: http://varunraiko.github.io/aggregate-functions/ Disclaimer: I hereby declare that all my work done so far (and future) by me on the Aggregate Functions’ project, is released under the NEW BSD licence. Regards, Varun

2 1

Re: [Maria-developers] Sachin weekly report
by Sachin Setia 24 Aug '16

24 Aug '16

Hi Sergei! Actually I completed the work on update and delete. Now they will use index for looking up records. But I am thinking I have done a lot of changes in optimizer which may break it , and also there are lots of queries where my code does not work, fixing this might take a long amount of time. I am thinking of a change in my existing code :- Suppose a table t1 create table t1 (a blob, b blob, c blob, unique(a,b,c)); In current code , for query like there will a KEY with only one keypart which points to field DB_ROW_HASH_1. It was okay for normal updates , insert and delete , but in the case of where optimization I have do a lot of stuff , first to match field (like in add_key_part), then see whether all the fields in hash_str are present in where or not, then create keys by calculating hash. I do this by checking the HA_UNIQUE_HASH flag in KEY , but this also makes (I think) optimizer code bad because of too much dependence. Also I need to patch get_mm_parts and get_mm_leaf function , which I think should not be patched. I am thinking of a another approach to this problem at server level instead of having just one keypart we can have 1+3 keypart. Last three keypart will be for field a, b, c and first one for DB_ROW_HASH_1 .These will be only at server level not at storage level. key_info->key_part will point at keypart containing field a , while key_part having field DB_ROW_HASH_1 will -1 index. By this way I do not have to patch more of optimizer code. But there is one problem , what should be the length of key_part? I am thinking of it equal to field->pack_length(), this would not work because while creating keys optimizer calls get_key_image() (which is real data so can exceed pack_lenght() in case of blob), so to get this work I have to patch optimizer where it calls get_key_image() and see if key is HA_UNIQUE_HASH . If yes then instead of get_key_image just use memcpy(key, field->ptr(), field->pack_length()); this wont copy the actual data, but we do not need actual data. I will patch handler methods like ha_index_read, ha_index_idx_read , multi_range_read_info_const basically handler methods which are related to index or range search. In these methods i need to calculate hash , which I can calculate from key_ptr but key_ptr doe not have actual data(in case of blobs etc).So to get the date for hash , I will make a field clone of (a,b,c etc) but there ptr will point in key_ptr. Then field->val_str() method will work simply and i can calculate hash. And also I can compare returned result with actual key in handler method itself. What do you think of this approach ? Regards sachin On Sat, Aug 20, 2016 at 11:16 PM, Sergei Golubchik <serg(a)mariadb.org> wrote: > Hi, Sachin! > > On Aug 19, Sachin Setia wrote: > > On Fri, Aug 19, 2016 at 2:42 PM, Sergei Golubchik <serg(a)mariadb.org> > wrote: > > > > > First. I believe you'll need to do your final evaluation soon, and > > > it will need to have a link to the code. Did you check google > > > guidelines about it? Is everything clear there? Do you need help > > > publishing your work in a format that google requires? > > > > > > They don't accept delays for any reasons, so even if your code is > > > not 100% complete and ready, you'd better still publish it and > > > submit the evaluation, because otherwise google will fail you and > > > that'd be too sad. > > > > > > If you'd like you can publish the google-way only the > > > unique-constraint part without further optimizer work. Or at least > > > please mention that you'd completed the original project and went > > > working on extensions. I mean, it's better than saying "the code is > > > not 100% complete" :) > > > > > Okay I am thinking of writing a blog post with a link to my github > > repository. > > Blog Link <http://sachin1001gsoc.blogspot.in/2016/08/gsoc-2016.html> > > Please check this. > > I think that'll do, yes. > > Regards, > Sergei > Chief Architect MariaDB > and security(a)mariadb.org >

2 3

Re: [Maria-developers] [Maria-discuss] Known limitation with TokuDB in Read Free Replication & parallel replication ?
by Kristian Nielsen 24 Aug '16

24 Aug '16

Rich Prohaska <prohaska7(a)gmail.com> writes: > On Tue, Aug 23, 2016 at 1:45 PM, Kristian Nielsen <knielsen(a)knielsen-hq.org> > wrote: >> In the original parallel replication patch, thd_report_wait_for() did not >> call back directly into tokudb_kill_query(). The kill happened >> asynchronously, in a separate background thread. Then there is no problem >> with TokuDB (or InnoDB) holding locks over the call to >> thd_report_wait_for(). >> >> I am considering re-introducing that orginal code - this might simplify >> TokuDB implementation (and would also simplify InnoDB implementation). >> I was never very happy about the way thd_report_wait_for() works currently. Ok, so I implemented this, available in this branch: https://github.com/knielsen/server/commits/toku_opr3 With this code, I am no longer able to reproduce any hangs with the tests I've been running so far. > BTW, the current_lock_tree_mutex logic is broken. The underlying tokuft > lock manager has one manager object (and its mutex) and many lock tree > objects (each with its own mutex). Since the thd_report_wait_for function > is called when holding the lock tree mutex (not the manager mutex), it can > be called in parallel; thus there is a race on the current_lock_tree_mutex > logic. Yes, and this is the main problem solved by doing the kill asynchronously. And from my tests, it looks like this was actually what was causing the problems - crash, assertion, and hangs. (I haven't determined this conclusively, but it seems at least plausible). It really was always an ugly hack around the locking problem with thd_report_wait_for() (InnoDB had a similar issue), it seems good to get rid of it. I still have the should_retry_lock_requests disabled in retry_all_lock_requests() - otherwise I get hangs. I haven't investigated this deeply yet. I also want to go back over the entire set of patches and see what needs cleaning up, and think about how this could go into 10.2 and possibly 10.1. - Kristian.

1 0

[Maria-developers] Commit emails reply-to maria-developers@lists.launchpad.net
by Sergey Vojtovich 24 Aug '16

24 Aug '16

Hi! Why do commit emails have "Reply-To: maria-developers(a)lists.launchpad.net"? I believe it is wrong at least for the following reasons: - I must be careful if I want to reply privately, e.g. something like "hold on, I'll review it in a few hours" doesn't deserve to hit maria-developers@ - I want a person, I'm replying to, to be primary recepient, but Reply-To substitutes his email with maria-devlopers@. Makes it easy to miss reviews. Sergei mentioned that one of reasons might be so that we want to keep commits(a)mariadb.org. I believe it doesn't work because: - if I do group reply, I end up with email that has CC: commits(a)mariadb.org Thanks, Sergey

1 0