- developers - lists.mariadb.org

[Maria-developers] Fwd: Gsoc 2016 Mdev 371 Unique index for blob
by Sachin Setia 29 Mar '16

29 Mar '16

Hi Sergei Did you see the patch.Actually the patch works for every combination of blob and varchars table definition. Can you please tell what extra modification I should make Regards sachin On Mon, Mar 28, 2016 at 9:34 PM, Sachin Setia <sachinsetia1001(a)gmail.com> wrote: > > ---------- Forwarded message ---------- > From: Sachin Setia <sachinsetia1001(a)gmail.com> > Date: Mon, Mar 28, 2016 at 9:13 PM > Subject: Re: [Maria-developers] Gsoc 2016 Mdev 371 Unique index for blob > To: maria-developers(a)lists.launchpad.net > > > I am Sorry I did not see this mail.So I did not send the prototype link in > proposal > > On Thu, Mar 24, 2016 at 10:41 PM, Sergei Golubchik <serg(a)mariadb.org> > wrote: > >> Hi, Sachin! >> >> As you might have noticed on the mailing list, there is another student >> who wants to do this project. >> >> It would help us choose, if you'd put your prototype code on github and >> put the link to the repository into your final proposal. >> >> Thanks! >> >> On Mar 15, Sachin Setia wrote: >> > Dear Developers, >> > >> > I was doing prototype for this project as Mr Sergei Golubchik suggested >> > >> >> Regards, >> Sergei >> Chief Architect MariaDB >> and security(a)mariadb.org >> > > >

2 1

[Maria-developers] Getting restarted with mariadb and a port to AIX
by Michael Felt 28 Mar '16

28 Mar '16

Hi all, This week I added a few issues that I felt were clear enough to report directly. This one I am not so sure of as it concerns two areas I know too little about: a) cmake (how to get an equivalent of GNU autotools ./configure --help) b) cmake (how to make it show the commands it is actually running (so I could add -bnoquiet and/or -bloadmap:XXX.map) c) 'strategic' grepping through C++ sources to find where an undefined label is calling something. After I 'patch' aka bandaid mariadb-5.5.48.0/sql-common/mysql_async.c (see https://jira.mariadb.org/browse/MDEV-9814) I get loader errors - and I cannot, in ways I could with an autotools generated 'make' see how to debug. Q) are there variables (e.g., V=1) I can pass to make so that the commands it performs are more verbose Q) have any ideas why these symbols are undefined? Thanks, Michael

2 1

[Maria-developers] Unique Index for blob
by Sachin Setia 28 Mar '16

28 Mar '16

Hello Devlopers Hi this is sachin. I applied for gsoc 2016(applied by name sachin setiya) My topic was unique index for blob(MyISAM) But i already completed some part of it .it is working for blobs varchar for definitions like create table tbl(abc varchar(2000) , xyz blob ,unique(abc,xyz)); I have corrected the table2myisam function . For documentation you can refer to my gsoc proposal If am doing wrong please let me know. I am attaching the patch file TODO Second thing which i need to do is to tweak check_definition function to correctly check for definition currently I am commenting the function code Regards sachin

1 1

Re: [Maria-developers] Gsoc 2016 Mdev 371 Unique index for blob
by Sachin Setia 28 Mar '16

28 Mar '16

I am Sorry I did not see this mail.So I did not send the prototype link in proposal On Thu, Mar 24, 2016 at 10:41 PM, Sergei Golubchik <serg(a)mariadb.org> wrote: > Hi, Sachin! > > As you might have noticed on the mailing list, there is another student > who wants to do this project. > > It would help us choose, if you'd put your prototype code on github and > put the link to the repository into your final proposal. > > Thanks! > > On Mar 15, Sachin Setia wrote: > > Dear Developers, > > > > I was doing prototype for this project as Mr Sergei Golubchik suggested > > > > Regards, > Sergei > Chief Architect MariaDB > and security(a)mariadb.org >

1 0

[Maria-developers] Please review MDEV-6353 my_ismbchar() and my_mbcharlen() refactoring
by Alexander Barkov 28 Mar '16

28 Mar '16

Hi Sergei, Please review a partial patch for MDEV-6353. It removes my_mbcharlen() in all remaining pieces of the code, except LOAD DATA and SELECT INTO OUTFILE. I'll do LOAD DATA and SELECT INTO OUTFILE in a separate patch. Thanks.

1 0

Re: [Maria-developers] Google Summer of Code 2016
by Soham Mankad 26 Mar '16

26 Mar '16

Respected sir, I am Soham Mankad and thanks Sergei sir for directing to me this developers list. As I had mailed you before I have gone through the basic requirements and goals of the project "Aggregate Stored Functions" and also thinking on varoius syntax and languages that can be used like hsql, Postgresql, or mysql. Also as you had mentioned in a reply to Vikas Gupta still i have not found any data base with this facility existing uptill now. Also I have seen the syntax and possible ways given by you in knowledge base of the project. The most basic form that you have mentioned i.e using nested commands i.e returning sum adding two numbers. I am also learning about various aspects of create function commands in different languages loke read only or returning which rows or coloumns so that optimum result will be given. I have submitted my proposal for this project on Gsoc'16 website. I would definitely appreciate an opportunity to work on this project. Regards, Soham Mankad.

1 0

[Maria-developers] Window functions: why some functions are not in select_lex->window_funcs?
by Sergey Petrunia 23 Mar '16

23 Mar '16

Hello Igor, I was trying to move JOIN::process_window_functions() from using join->fields_list to using join->select_lex->window_funcs. And I discovered that join->select_lex->window_funcs does not contain window function items that use window definition. Is this intentional? Example: create table t0 (a int); insert into t0 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); create table t1 (pk int, c int); insert into t1 select a+1,1 from t0; update t1 set c=2 where pk not in (1,2,3,4); select pk, c, count(*) over (partition by c order by pk rows between 2 preceding and 2 following) as CNT from t1 Here: #1 0x0000555555af5db6 in AGGR_OP::end_send (this=0x7fff4c0086c8) (gdb) p join->select_lex->window_funcs $69 = {<base_list> = {<Sql_alloc> = {<No data fields>}, first = 0x7fff4c005d18, last = 0x7fff4c005d18, elements = 1}, <No data fields>} But: select pk, c, count(*) over w1 as CNT from t1 window w1 as (partition by c order by pk rows between 2 preceding and 2 following); (gdb) p join->select_lex->window_funcs $74 = {<base_list> = {<Sql_alloc> = {<No data fields>}, first = 0x555556d48820, last = 0x55555ab10b70, elements = 0}, <No data fields>} BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 1

Re: [Maria-developers] Gsoc 2016 Mdev 371 Unique index for blob
by Sachin Setia 23 Mar '16

23 Mar '16

Hello Sergei Actually I was prototyping for blob and varchar for aria and myisam storage engine. My prototype worked for complex definations like craete table(abc int primary key, blob_col blob unique, varchar_col varchar(1000) unique) engine=myisam; Solved the issue of frm file incosistance. As you suggested for doing it for innodb i am current working on it.Innodb does not natively support hash based index. when we run select distinct column from tbl; it use create_internal_tmp_table() which uses maria storage engine for creating tmp table. But query like this works MariaDB [sachin]> create table iu2(abc blob unique); Query OK, 0 rows affected (0.04 sec) MariaDB [sachin]> insert into iu2 values(1); Query OK, 1 row affected (0.03 sec) MariaDB [sachin]> insert into iu2 values(1); ERROR 1062 (23000): Duplicate entry '1' for key 'abc' this query does not use hash but it simply compares values Will write a proposal shortly. On Tue, Mar 22, 2016 at 4:20 PM, Sachin Setia <sachinsetia1001(a)gmail.com> wrote: > Hello Sergi > Actually I was prototyping for blob and varchar for aria and myisam > storage engine. > My prototype worked for complex definations like > craete table(abc int primary key, blob_col blob unique, varchar_col > varchar(1000) unique) engine=myisam; > Solved the issue of frm file incosistance. > > As you suggested for doing it for innodb i am current working on it.Innodb > does not natively support hash based index. > when we run select distinct column from tbl; > it use create_internal_tmp_table() which uses maria storage engine for > creating tmp table. > But query like this works > MariaDB [sachin]> create table iu2(abc blob unique); > Query OK, 0 rows affected (0.04 sec) > > MariaDB [sachin]> insert into iu2 values(1); > Query OK, 1 row affected (0.03 sec) > > MariaDB [sachin]> insert into iu2 values(1); > ERROR 1062 (23000): Duplicate entry '1' for key 'abc' > this query does not use hash but it simply compares values > Will write a proposal shortly. > > > Regards > sachin > > On Sat, Mar 19, 2016 at 1:52 AM, Sergei Golubchik <serg(a)mariadb.org> > wrote: > >> Hi, Sachin! >> >> On Mar 18, Sachin Setia wrote: >> > >> > ERROR 1030 (HY000): Got error 190 "Incompatible key or row definition >> > between the MariaDB .frm file and the information in the storage engine. >> > You have to dump an" from storage engine MyISAM >> > >> > We are getting this becuase in mi_create for each unique_def it creates >> one >> > keydef and writes it.And this creates two problem >> > 1. frm keydef algorithm is different from saved kefdef algorithm(always >> > zero) for the time I have solved this problem . >> > >> > 2.Mismatch between keydef's keysegs the reason for this is when >> mi_create >> > creates keyseg for unique_def it did not keeps the orignal uniquedef's >> > keyseg parameters in mind like language start length which creates >> problem >> > in check_definition function in ha_myisam.cc.I am currently working on >> it >> > Once again sorry for this foolish mistake. >> > Regars >> > sachin >> >> No problem, everyone makes mistakes :) >> >> It's a prototype, after all. It's much more important that you >> understand how the code works and why it doesn't work. >> >> Regards, >> Sergei >> Chief Architect MariaDB >> and security(a)mariadb.org >> > >

2 3

[Maria-developers] More memory allocation for key seg
by Sachin Setia 23 Mar '16

23 Mar '16

Hello Devlopers I was debugging the source code of mariadb. My query was MariaDB [sachin]> create table pro2(abc int primary key,xyz int , ass int ,unique(xyz,ass))engine=myisam; In table2myisam function of file ha_myisam.cc i found that we are allocating more memory for keysegments if (!(my_multi_malloc(MYF(MY_WME), recinfo_out, (share->fields * 2 + 2) * sizeof(MI_COLUMNDEF), keydef_out, share->keys * sizeof(MI_KEYDEF), &keyseg, (share->key_parts + share->keys) * sizeof(HA_KEYSEG), ^ here why this NullS))) And we donot use these extra key segments because suceeding for loop use only total number of share->key_parts keysegments. And same in ha_maria file.Is this a bug or I am missing something Regards sachin

2 2

[Maria-developers] Please review MDEV-9604 crash in Item::save_in_field with empty enum value
by Alexander Barkov 22 Mar '16

22 Mar '16

Hi Sergei, It fixes the reported crash, and additionally inconsistencies when comparing temporal columns to empty strings: In some cases it returned rows, in other cases it returned no rows, depending on: - the engine - the type of comparison arguments (columns vs literals). - presence of an index Now everything works in the same way. Note, this additional problem is not 10.1 specific, it existed in 10.0. But 10.0 and 10.1 worked differently in certain cases. Thanks!

2 4

Re: [Maria-developers] Aggregate stored functions [MDEV-7773]
by Sergei Golubchik 22 Mar '16

22 Mar '16

Hi, Varun! On Mar 21, Varun Gupta wrote: > Hi Sergei, > > 4) :create an Item_sum_sp to be able to use aggregate stored functions in > queries. > Does this mean we can have the aggregate stored functions in the SELECT , > WHERE etc clauses?can you give me more details about this. Yes, of course. That's how functions are used, right? :) SELECT AVG(x), COUNT(y), MAX(z) FROM t1 WHERE aaa=12 GROUP BY bbb; for example. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 3

Re: [Maria-developers] 29e99a7: MDEV-717 LP:1003679 - Wrong binlog order on concurrent DROP schema and
by Sergei Golubchik 21 Mar '16

21 Mar '16

Hi, Alexey! In general the patch is pretty good. But I've had few comments, mostly about bad choice of error messages. It's a historical behavoir that you decided not to change. If you'd prefer to leave it as is, that's ok, but then, please, create a new MDEV about these error messages. And add test cases :) On Mar 20, Alexey Botchkov wrote: > revision-id: 29e99a701817da56499df9126830d9533e4d66f3 (mariadb-10.1.12-26-g29e99a7) > parent(s): 7cb16dc2a369058b31df1b3999989f6beff94d07 > committer: Alexey Botchkov > timestamp: 2016-03-20 23:57:26 +0400 > message: > > MDEV-717 LP:1003679 - Wrong binlog order on concurrent DROP schema and > CREATE function. > The cause of the issue is when DROP DATABASE takes > metadata lock and is in progress through it's > execution, a concurrently running CREATE FUNCTION checks > for the existence of database which it succeeds and then it > waits on the metadata lock. Once DROP DATABASE writes to > BINLOG and finally releases the metadata lock on schema > object, the CREATE FUNCTION waiting on metadata lock gets > in it's code path and succeeds and writes to binlog. Please structure the commit comment as <first line - the subject> <empty line> <the rest - the body> That's the convention and various tools, for example, github itself, use it to parse and display the commit subject and body. > --- > sql/events.cc | 29 ++++++++++++++------ > sql/sp.cc | 81 +++++++++++++++++++++++++++++++++++++------------------- > sql/sp.h | 2 +- > sql/sql_parse.cc | 56 +++++++-------------------------------- What, no test cases at all? Please, add tests for everything you've fixed: - create event - update event with moving to a new database > diff --git a/sql/sp.cc b/sql/sp.cc > index 6ec5914..11ba186 100644 > --- a/sql/sp.cc > +++ b/sql/sp.cc > @@ -1040,7 +1045,22 @@ sp_create_routine(THD *thd, stored_procedure_type type, sp_head *sp) > > /* Grab an exclusive MDL lock. */ > if (lock_object_name(thd, mdl_type, sp->m_db.str, sp->m_name.str)) > - DBUG_RETURN(SP_OPEN_TABLE_FAILED); > + { > + my_error(ER_SP_STORE_FAILED, MYF(0), SP_TYPE_STRING(type), sp->m_name.str); No better error message here? May be ER_BAD_DB_ERROR? I understand that the old behavior is ER_SP_STORE_FAILED here. > + DBUG_RETURN(TRUE); > + } > + > + /* > + Check that a database directory with this name > + exists. Design note: This won't work on virtual databases > + like information_schema. > + */ > + if (check_db_dir_existence(sp->m_db.str)) > + { > + my_error(ER_BAD_DB_ERROR, MYF(0), sp->m_db.str); > + DBUG_RETURN(TRUE); > + } > + > > /* Reset sql_mode during data dictionary operations. */ > thd->variables.sql_mode= 0; > @@ -1049,7 +1069,9 @@ sp_create_routine(THD *thd, stored_procedure_type type, sp_head *sp) > thd->count_cuted_fields= CHECK_FIELD_WARN; > > if (!(table= open_proc_table_for_update(thd))) > - ret= SP_OPEN_TABLE_FAILED; > + { > + my_error(ER_SP_STORE_FAILED, MYF(0), SP_TYPE_STRING(type),sp->m_name.str); better error message, may be? > + } > else > { > /* Checking if the routine already exists */ > @@ -1268,7 +1289,8 @@ sp_create_routine(THD *thd, stored_procedure_type type, sp_head *sp) > &(thd->lex->definer->host), > saved_mode)) > { > - ret= SP_INTERNAL_ERROR; > + my_error(ER_SP_STORE_FAILED, MYF(0), > + SP_TYPE_STRING(type), sp->m_name.str); That's OOM, can be a better error message too. Btw, could you reverse the return value of show_create_sp() please? > goto done; > } > /* restore sql_mode when binloging */ > @@ -1277,9 +1299,14 @@ sp_create_routine(THD *thd, stored_procedure_type type, sp_head *sp) > if (thd->binlog_query(THD::STMT_QUERY_TYPE, > log_query.ptr(), log_query.length(), > FALSE, FALSE, FALSE, 0)) > - ret= SP_INTERNAL_ERROR; > + { > + my_error(ER_SP_STORE_FAILED, MYF(0), > + SP_TYPE_STRING(type), sp->m_name.str); that's very confusing. "Failed to CREATE %s %s" when the creation has actually succeeded. The same applies to the OOM error above. > + goto done; > + } > thd->variables.sql_mode= 0; > } > + ret= FALSE; Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] Randomness in TokuDB builds
by Otto Kekäläinen 21 Mar '16

21 Mar '16

Hello TokuDB experts! There is some randomness, maybe some unique build id in every TokuDB build that breaks the reproducable builds of it in Debian. Check out: https://tests.reproducible-builds.org/dbd/unstable/amd64/mariadb-10.0_10.0.… I can't find any mentions of a build id or similar in the sources. Can somebody help me figure out where the randomness stems from, and how I should patch the source to make the build deterministic? - Otto

2 1

[Maria-developers] Please review MDEV-9516 type error when setting session variable
by Alexander Barkov 21 Mar '16

21 Mar '16

Hello Sergei, Please review a patch for MDEV-9516. Thanks.

1 0

Re: [Maria-developers] 8d675d5: MDEV-6058 MySQL Bug #11766693: LOG-SLOW-ADMIN-STATEMENTS AND
by Sergei Golubchik 21 Mar '16

21 Mar '16

Hi, Alexey! On Mar 21, Alexey Botchkov wrote: > revision-id: 8d675d5764e7d8ecad19fafdd167bee9a41d1dc7 (mariadb-10.1.12-27-g8d675d5) > parent(s): 94768542115289272996f6450b7d3cafa75dcead > committer: Alexey Botchkov > timestamp: 2016-03-21 13:02:09 +0400 > message: > > MDEV-6058 MySQL Bug #11766693: LOG-SLOW-ADMIN-STATEMENTS AND > LOG-SLOW-SLAVE-STATEMENTS NOT DISPLAYED. > These parameters were moved from the command line options to the > system variables section. Treatment of the > opt_log_slow_slave_statements changed to let the > dynamic change of the variable. Thanks! Looks good, just one question: > diff --git a/mysql-test/suite/rpl/r/rpl_slow_query_log.result b/mysql-test/suite/rpl/r/rpl_slow_query_log.result > index 79b83b4..871722e 100644 > --- a/mysql-test/suite/rpl/r/rpl_slow_query_log.result > +++ b/mysql-test/suite/rpl/r/rpl_slow_query_log.result > @@ -76,6 +76,18 @@ ALTER TABLE t1 ADD INDEX id1(a); > INSERT INTO t1 values(1, sleep(3)); > ### Assertion is good. Both Master and Slave exhibit the > ### same number of queries in slow log: 1 > +******************************************************************** > +**** TRUNCATE the slow log then check whether runtime changes of > +**** log_slow_slave_statements work without slave restart. > +******************************************************************** > +SET @old_log_slow_slave_statements= @@global.log_slow_slave_statements; > +SET @@global.log_slow_slave_statements = off; > +TRUNCATE mysql.slow_log; > +INSERT INTO t1 values(1, sleep(3));; > +0 > +SET @@global.log_slow_slave_statements = on; > +INSERT INTO t1 values(1, sleep(3));; > +1 > SET @@global.log_output= @old_log_output; > SET @@global.long_query_time= @old_long_query_time; > DROP TABLE t1; > diff --git a/mysql-test/suite/rpl/t/rpl_slow_query_log.test b/mysql-test/suite/rpl/t/rpl_slow_query_log.test > index faf037a..3505883 100644 > --- a/mysql-test/suite/rpl/t/rpl_slow_query_log.test > +++ b/mysql-test/suite/rpl/t/rpl_slow_query_log.test > @@ -299,6 +299,39 @@ if ($master_slow_query == $slave_slow_query) > -- echo ### same number of queries in slow log: $master_slow_query > } > > +-- echo ******************************************************************** > +-- echo **** TRUNCATE the slow log then check whether runtime changes of > +-- echo **** log_slow_slave_statements work without slave restart. > +-- echo ******************************************************************** > + > +SET @old_log_slow_slave_statements= @@global.log_slow_slave_statements; > +SET @@global.log_slow_slave_statements = off; > +TRUNCATE mysql.slow_log; > + > +-- connection master > + > +--disable_warnings > +-- eval $slow_query; > +--enable_warnings > +sync_slave_with_master; > + > +-- connection slave > + > +eval SELECT sql_text FROM mysql.slow_log WHERE sql_text like '$slow_query'; > + > +SET @@global.log_slow_slave_statements = on; > + > +-- connection master > + > +--disable_warnings > +-- eval $slow_query; > +--enable_warnings > +sync_slave_with_master; > + > +-- connection slave > + > +eval SELECT sql_text FROM mysql.slow_log WHERE sql_text like '$slow_query'; Did you run this test? It seems the result doesn't match the test. The result looks like as if you'd have eval SELECT COUNT(*) FROM ... Please, run the full test suite before pushing. > + > -- connection master > SET @@global.log_output= @old_log_output; > SET @@global.long_query_time= @old_long_query_time; Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] MDEV-9369 IN operator with ( num, NULL ) gives inconsistent result
by Sergei Golubchik 21 Mar '16

21 Mar '16

Hi, Alexander! On Feb 08, Alexander Barkov wrote: > Hello Sergei, > > Please review a patch for MDEV-9369. > > Briefly, it fixes cmp_item::cmp() to return three possible values: > FALSE, TRUE, UNKNOWN. These values are then recursively pulled to > the very top level, to the owner Item_xxx. > > The owner Item_func_in (as well as Item_func_case and Item_func_equal) > then further "translate" these tree-values logic into the traditional > val_int()+null_value combination using some additional information. > > For example, Item_func_in::val_int() uses "negated" as an additional > information. > > The patch looks Ok for me. Looks ok to me too. A couple of stylistic comments, see below. ok to push afterwards. > diff --git a/sql/item_cmpfunc.cc b/sql/item_cmpfunc.cc > index d5f5087..39003de 100644 > --- a/sql/item_cmpfunc.cc > +++ b/sql/item_cmpfunc.cc > @@ -3837,15 +3843,15 @@ void cmp_item_decimal::store_value(Item *item) > /* val may be zero if item is nnull */ > if (val && val != &value) > my_decimal2decimal(val, &value); > + set_null_value(item->null_value); > } > > > int cmp_item_decimal::cmp(Item *arg) > { > my_decimal tmp_buf, *tmp= arg->val_decimal(&tmp_buf); > - if (arg->null_value) > - return 1; > - return my_decimal_cmp(&value, tmp); > + return (m_null_value || arg->null_value) ? if you access m_null_value directly, you don't need a setter for it. it'll be less confusing if you replace set_null_value(X) with m_null_value=X. alternatively, you can add a getter is_null_value(). But I personally wouldn't do it, cmp_xxx is a small set of simple classes, no need to get too enterprisey there. > + UNKNOWN : (my_decimal_cmp(&value, tmp) != 0); > } > > > @@ -3892,10 +3900,10 @@ cmp_item *cmp_item_datetime::make_same() > } > > > -bool Item_func_in::nulls_in_row() > +bool Item_func_in::list_contains_null() I found the old name clear enough. But ok, whatever... > { > Item **arg,**arg_end; > - for (arg= args+1, arg_end= args+arg_count; arg != arg_end ; arg++) > + for (arg= args + 1, arg_end= args+arg_count; arg != arg_end ; arg++) > { > if ((*arg)->null_inside()) > return 1; > @@ -4010,6 +4018,32 @@ void Item_func_in::fix_length_and_dec() > } > } > > + /* > + First conditions for bisection to be possible: > + 1. All types are similar, and > + 2. All expressions in <in value list> are const > + */ > + bool bisection_possible= > + type_cnt == 1 && // 1 > + const_itm; // 2 > + if (bisection_possible) > + { > + /* > + In the presence of NULLs, the correct result of evaluating this item > + must be UNKNOWN or FALSE. To achieve that: > + - If type is scalar, we can use bisection and the "have_null" boolean. > + - If type is ROW, we will need to scan all of <in value list> when > + searching, so bisection is impossible. Unless: > + 3. UNKNOWN and FALSE are equivalent results > + 4. Neither left expression nor <in value list> contain any NULL value > + */ > + > + if (cmp_type == ROW_RESULT && > + !((is_top_level_item() && !negated) || // 3 > + (!list_contains_null() && !args[0]->maybe_null))) // 4 > + bisection_possible= false; I think the condition will be easier to understand if you remove the negation of that complex condition in parentheses: if (cmp_type == ROW_RESULT && ((!is_top_level_item() || negated) && // 3 (list_contains_null() || args[0]->maybe_null))) // 4 I find this ^^^ much easier to understand. > + } > + > if (type_cnt == 1) > { > if (cmp_type == STRING_RESULT && Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 1

[Maria-developers] Google Summer of Code [MySql -> MsSql]
by Bo Zhu 20 Mar '16

20 Mar '16

Hi, everyone. I am Bo, a second year master student in Clemson university, SC. I had ever implemented a small relational database and java compiler. I am really interested in the MaxScale filter to translate the Mysql query to MsSql one. In this task, we need to write some C++ code as a MaxScale's filter, right ? I think it is about syntax translation which is a compiler front-end problem. I have shared my proposal draft and am really looking forward to get some feedback.

1 0

[Maria-developers] MaxScale filter to real Microsoft SQL Server syntax
by Wepngong Benaiah 20 Mar '16

20 Mar '16

> > Hello everyone, I am Wepngong Ngeh Benaiah, a 4th year Computer Engineering student at the University of Buea, Cameroon. I have been looking at gsoc projects on https://mariadb.com/kb/en/mariadb/google-summer-of-code-2016/ and got interested in *MaxScale filter to real Microsoft SQL Server syntax [https://jira.mariadb.org/browse/MXS-1 <https://jira.mariadb.org/browse/MXS-1>]. * I will love to contribute to MariaDB by means of GSOC2016. I have drafted a proposal and submitted to google and will love to get feedback on this Thanks in advance -- Buea,Cameroon University of Buea Faculty of Engineering and Technology Computer Engineering Software Engineering * Wepngong Ngeh Benaiah* *https://github.com/bwepngong <https://github.com/bwepngong>*

1 0

[Maria-developers] GSoC 2016 CONC-125
by Christian Massey 19 Mar '16

19 Mar '16

Hi everyone, I am Christian Massey a computer science student at York University and I wish to work on Providing import and export functions for popular data formats like JSON, XML , PHP, ... for Connector/C and MariaDB Server. I have been inspecting the code to understand how the dynamic column packed string is encoded and I see kind on big endian encoding( Correct me if I am wrong please). Also I see that the old format already work like arrays and it uses 3 bits for encoding type which is the main reason why not all types in the enum are supported in this old format. DBUG_ASSERT(((type - 1) & (~7)) == 0); /* fit in 3 bits */ (Again please correct me if I am wrong. The encoding is not easy to understand) My Question is do we need to create a completely new type to support arrays or do we change the old format which already work like arrays in a way but lacks support for some types(I don't know which of the enums yet). Please, I wish to write a proposal for this project and hope to discuss more with mentors for more clarification on how we are going to implement this as it seems to be a very important if not the most critical part of this project. Cheers, Kris.

2 2

[Maria-developers] Aggregate stored functions [MDEV-7773]
by Varun Gupta 19 Mar '16

19 Mar '16

Well i have checked out the syntax for the databases for aggregate functions. I thought using the syntax similar to that of HSQL would be good. In it if we are having N tuples then we are computing the values for the N rows and then after that we make another call and return the value for the function. Well this seems very reasonable to do. Using PostgreSQL <http://www.postgresql.org/docs/8.3/static/sql-createaggregate.html> we need to have two regular functions but if we have an aggregate function like Count(column_name) , then this would be done in just a single function , I guess we would not need two functions for it . Regards, Varun

2 11

Re: [Maria-developers] b1a1e5b: MDEV-6058 MySQL Bug #11766693: LOG-SLOW-ADMIN-STATEMENTS AND LOG-SLOW-SLAVE-STATEMENTS NOT DISPLAYED.
by Sergei Golubchik 18 Mar '16

18 Mar '16

Hi, Alexey! On Mar 18, Alexey Botchkov wrote: > revision-id: b1a1e5b3d3d5ab95c596580de07daf38a3efa039 (mariadb-10.1.12-19-gb1a1e5b) > parent(s): 4aac51db9a55a11574cbfa60484d4329d35b6f2c > committer: Alexey Botchkov > timestamp: 2016-03-18 11:47:05 +0400 > message: > > MDEV-6058 MySQL Bug #11766693: LOG-SLOW-ADMIN-STATEMENTS AND LOG-SLOW-SLAVE-STATEMENTS NOT DISPLAYED. > These parameters were moved from the command line options to the > system variables section. Treatment of the > opt_log_slow_slave_statements changed to let the dynamic change > of the variable. > > --- > mysql-test/r/variables.result | 10 ++ > mysql-test/suite/rpl/r/rpl_slow_query_log.result | 10 ++ > mysql-test/suite/rpl/t/rpl_slow_query_log.test | 43 ++++++++ > .../r/log_slow_admin_statements_basic.result | 76 ++++++++++++++ > .../r/log_slow_admin_statements_func.result | 46 +++++++++ > .../r/log_slow_slave_statements_basic.result | 76 ++++++++++++++ > .../t/log_slow_admin_statements_basic.test | 110 +++++++++++++++++++++ > .../sys_vars/t/log_slow_admin_statements_func.test | 61 ++++++++++++ > .../t/log_slow_slave_statements_basic.test | 110 +++++++++++++++++++++ > mysql-test/t/variables.test | 11 +++ I suppose you didn't run the full test suite. Otherwise you would've noticed that sysvar_* results were changed. Also, because we have sysvar_* tests, we no longer create <varname>_basic tests for every new variable, they are no longer useful. Starting from 10.1 you only need to test the functionality of new variables, not their basic properties. > sql/log_event.cc | 10 ++ > sql/mysqld.cc | 8 -- > sql/sys_vars.cc | 13 +++ > 13 files changed, 576 insertions(+), 8 deletions(-) > > diff --git a/mysql-test/r/variables.result b/mysql-test/r/variables.result > index fef3e4a..e46e65f 100644 > --- a/mysql-test/r/variables.result > +++ b/mysql-test/r/variables.result > @@ -1797,3 +1797,13 @@ select * from information_schema.session_variables where variable_name='sql_mode > VARIABLE_NAME VARIABLE_VALUE > SQL_MODE ANSI_QUOTES > End of 5.5 tests > +show variables like 'log_slow%statements'; > +Variable_name Value > +log_slow_admin_statements OFF > +log_slow_slave_statements OFF > +select * from information_schema.global_variables where variable_name like 'log_slow_admin_statements'; > +VARIABLE_NAME VARIABLE_VALUE > +LOG_SLOW_ADMIN_STATEMENTS OFF > +select * from information_schema.global_variables where variable_name like 'log_slow_slave_statements'; > +VARIABLE_NAME VARIABLE_VALUE > +LOG_SLOW_SLAVE_STATEMENTS OFF You can remove this test. > diff --git a/mysql-test/suite/rpl/t/rpl_slow_query_log.test b/mysql-test/suite/rpl/t/rpl_slow_query_log.test > index faf037a..de336fc 100644 > --- a/mysql-test/suite/rpl/t/rpl_slow_query_log.test > +++ b/mysql-test/suite/rpl/t/rpl_slow_query_log.test > @@ -299,6 +299,49 @@ if ($master_slow_query == $slave_slow_query) > -- echo ### same number of queries in slow log: $master_slow_query > } > > +-- echo ******************************************************************** > +-- echo **** TRUNCATE the slow log then check whether runtime changes of > +-- echo **** log_slow_slave_statements work without slave restart. > +-- echo ******************************************************************** > + > +SET @old_log_slow_slave_statements= @@global.log_slow_slave_statements; > +SET @@global.log_slow_slave_statements = off; > +TRUNCATE mysql.slow_log; > + > +-- connection master > + > +--disable_warnings > +-- eval $slow_query; > +--enable_warnings > +sync_slave_with_master; > + > +-- connection slave > + > +let $slave_slow_query= `SELECT count(*) = 1 FROM mysql.slow_log WHERE sql_text like '$slow_query'`; > +if ($slave_slow_query) > +{ > + SELECT * FROM mysql.slow_log; > + die "Assertion failed! Slow query FOUND in slow query log. Bailing out!"; > +} please, NO! If you want to test whether a certain row is present in a certain table you do SELECT columns FROM table; may be with a WHERE clause. But never if (`SELECT`) { die; } > + > +SET @@global.log_slow_slave_statements = on; > + > +-- connection master > + > +--disable_warnings > +-- eval $slow_query; > +--enable_warnings > +sync_slave_with_master; > + > +-- connection slave > + > +let $slave_slow_query= `SELECT count(*) = 1 FROM mysql.slow_log WHERE sql_text like '$slow_query'`; > +if (!$slave_slow_query) > +{ > + SELECT * FROM mysql.slow_log; > + die "Assertion failed! Slow query NOT FOUND in slow query log. Bailing out!"; > +} Same. > + > -- connection master > SET @@global.log_output= @old_log_output; > SET @@global.long_query_time= @old_long_query_time; > diff --git a/mysql-test/suite/sys_vars/r/log_slow_admin_statements_basic.result b/mysql-test/suite/sys_vars/r/log_slow_admin_statements_basic.result > new file mode 100644 > index 0000000..1cd9135 > --- /dev/null > +++ b/mysql-test/suite/sys_vars/r/log_slow_admin_statements_basic.result Remove this useless test, please > diff --git a/mysql-test/suite/sys_vars/r/log_slow_slave_statements_basic.result b/mysql-test/suite/sys_vars/r/log_slow_slave_statements_basic.result > new file mode 100644 > index 0000000..b9e88ce > --- /dev/null > +++ b/mysql-test/suite/sys_vars/r/log_slow_slave_statements_basic.result > @@ -0,0 +1,76 @@ And this one, too. > diff --git a/mysql-test/suite/sys_vars/t/log_slow_admin_statements_func.test b/mysql-test/suite/sys_vars/t/log_slow_admin_statements_func.test > new file mode 100644 > index 0000000..78e4d35 > --- /dev/null > +++ b/mysql-test/suite/sys_vars/t/log_slow_admin_statements_func.test > + > +# ALTER, OPTIMIZE, CHECK and DROP operations should be logged in slow query log. > +SELECT sql_text FROM mysql.slow_log WHERE sql_text LIKE '%TABLE log_slow_admin_statements%'; this is how one checks whether a query was logged. > diff --git a/sql/log_event.cc b/sql/log_event.cc > index e99ef16..9793f7d 100644 > --- a/sql/log_event.cc > +++ b/sql/log_event.cc > @@ -4428,6 +4428,16 @@ int Query_log_event::do_apply_event(rpl_group_info *rgi, > if (thd->m_digest != NULL) > thd->m_digest->reset(thd->m_token_array, max_digest_length); > > + if (thd->slave_thread && > + (thd->variables.sql_log_slow != opt_log_slow_slave_statements)) > + { > + /* > + The log_slow_slave_statements variable can be changed dynamically, > + so we have to set the sql_log_slow respectively. > + */ > + thd->variables.sql_log_slow= opt_log_slow_slave_statements; > + } Really? Why not if (thd->slave_thread) thd->variables.sql_log_slow= opt_log_slow_slave_statements; Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 2

[Maria-developers] failing tests
by Alexander Barkov 18 Mar '16

18 Mar '16

Hello Olivier, a few tests are failing in the main 10.1 tree on Linux: connect.xml connect.xml_grant connect.xml_html connect.xml_mdev5261 connect.xml_mult Can you please have a look? Thanks!

2 1

[Maria-developers] Please review MDEV-9653 Assertion `length || !scale' failed in uint my_decimal_length_to_precision(uint, uint, bool)
by Alexander Barkov 18 Mar '16

18 Mar '16

Hello Sergei, Please review a patch for: MDEV-9653 Assertion `length || !scale' failed in uint my_decimal_length_to_precision(uint, uint, bool) Thanks!

2 6

Re: [Maria-developers] 72b709a: MDEV-9701: CREATE VIEW with GROUP BY or ORDER BY and constant produces invalid definition
by Sergei Golubchik 17 Mar '16

17 Mar '16

Hi, Sanja! On Mar 16, OleksandrByelkin wrote: > revision-id: 72b709ac7503ae6dd2b5e9049322fefb90b0ebbe (mariadb-10.1.12-16-g72b709a) > parent(s): 9b53d84d14a9b031d193f6beae382a232aa738e3 > committer: Oleksandr Byelkin > timestamp: 2016-03-16 19:49:17 +0100 > message: > > MDEV-9701: CREATE VIEW with GROUP BY or ORDER BY and constant produces invalid definition > > Fixed printing integer constant in the ORDER clause (MySQL solution) > Removed workaround for double resolving counter in the ORDER. > > --- > mysql-test/r/view.result | 19 +++++++++++++++++++ > mysql-test/t/view.test | 17 +++++++++++++++++ > sql/sql_lex.cc | 32 ++++++++++++-------------------- > sql/sql_select.cc | 6 +++++- > sql/sql_union.cc | 11 +++++++++++ > 5 files changed, 64 insertions(+), 21 deletions(-) > > diff --git a/mysql-test/r/view.result b/mysql-test/r/view.result > index 8ee72c3..d4a2a9b 100644 > diff --git a/sql/sql_lex.cc b/sql/sql_lex.cc > index fd84109..31fc5f9 100644 > --- a/sql/sql_lex.cc > +++ b/sql/sql_lex.cc > @@ -2689,30 +2689,22 @@ void st_select_lex::print_order(String *str, > { > if (order->counter_used) > { > - if (!(query_type & QT_VIEW_INTERNAL)) > - { > char buffer[20]; > size_t length= my_snprintf(buffer, 20, "%d", order->counter); > str->append(buffer, (uint) length); you can replace these three lines with str->append_ulonglong(order->counter); > } > else > { > - /* replace numeric reference with expression */ > + /* replace numeric reference with equivalent for ORDER constant */ > if (order->item[0]->type() == Item::INT_ITEM && > order->item[0]->basic_const_item()) > { > - char buffer[20]; > - size_t length= my_snprintf(buffer, 20, "%d", order->counter); > - str->append(buffer, (uint) length); > /* make it expression instead of integer constant */ > - str->append(STRING_WITH_LEN("+0")); > + str->append(STRING_WITH_LEN("''")); > } > else > (*order->item)->print(str, query_type); > } > - } > - else > - (*order->item)->print(str, query_type); > if (!order->asc) > str->append(STRING_WITH_LEN(" desc")); > if (order->next) > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index fff24b9..f14e8b1 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -21874,7 +21874,11 @@ find_order_in_list(THD *thd, Item **ref_pointer_array, TABLE_LIST *tables, > */ > if (order_item->type() == Item::INT_ITEM && order_item->basic_const_item()) > { /* Order by position */ > - uint count= (uint) order_item->val_int(); > + uint count; > + if (order->counter_used) how can this happen? > + count= order->counter; // counter was once resolved > + else > + count= (uint) order_item->val_int(); > if (!count || count > fields.elements) > { > my_error(ER_BAD_FIELD_ERROR, MYF(0), > diff --git a/sql/sql_union.cc b/sql/sql_union.cc > index c835083..8145cec 100644 > --- a/sql/sql_union.cc > +++ b/sql/sql_union.cc > @@ -1161,11 +1161,22 @@ List<Item> *st_select_lex_unit::get_unit_column_types() > return &sl->item_list; > } > > + > +static void cleanup_order(ORDER *order) > +{ > + for (; order; order= order->next) > + order->counter_used= 0; > +} > + > + > bool st_select_lex::cleanup() > { > bool error= FALSE; > DBUG_ENTER("st_select_lex::cleanup()"); > > + cleanup_order(order_list.first); > + cleanup_order(group_list.first); why is it needed? > if (join) > { > DBUG_ASSERT((st_select_lex*)join->select_lex == this); Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 3

Re: [Maria-developers] [Commits] 1e542ec: MDEV-9382: After updating mariadb server apt-configure fails
by Sergei Golubchik 17 Mar '16

17 Mar '16

Hi, Nirbhay! Ok to push. And, please, check whether /etc/default/mariadb needs to be mentioned in the corresponding (mariadb on debian/ubuntu) knowledgebase pages. On Mar 16, Nirbhay Choubey wrote: > revision-id: 1e542ecd8af6f78db767a9b3da016b38d1ec3970 (mariadb-10.1.12-6-g1e542ec) > parent(s): b3bde93149e7135507fd7abbca2786d77552953d > author: Nirbhay Choubey > committer: Nirbhay Choubey > timestamp: 2016-03-16 19:39:19 -0400 > message: > > MDEV-9382: After updating mariadb server apt-configure fails > > When acting as a Galera receiver node, server startup may take > more than 30 secs (the current default) as it has to wait for > SST/IST operation to complete besides spending some time doing > wsrep recovery. > > Fixed by raising the default value of MYSQLD_STARTUP_TIMEOUT > to 60 secs. Also sourced /etc/default/mariadb into the init > script so that it can be used to set MYSQLD_STARTUP_TIMEOUT. > Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] NO PAD Collation for MariaDB
by Melissa Rose 17 Mar '16

17 Mar '16

Hello everyone, I am Melisa Rose from University of Leeds UK and I wish to contribute to MariaDB under GSoC 2016. I am interested in the project involving adding NO PAD collations to default character sets(MDEV-9711 <https://jira.mariadb.org/browse/MDEV-9711>). Apparently, this project does not require thorough understanding of the Server Internal code and the Instructions highlighted by Alexander Barkov on how to implement this are straight forward. I will be submitting a proposal for this project hoping i'm linked to the assigned mentor soon. Look forward to an awesome summer with MariaDB. Kind regards, Melisa Rose.

2 3

Re: [Maria-developers] [Maria-discuss] Collations, trailing spaces and unique indexes
by Alexander Barkov 17 Mar '16

17 Mar '16

Hi Binarus, On 03/16/2016 06:13 PM, Binarus wrote: >> I just wanted to split the task into these steps: >> >> 1. Default and _bin collations. >> This is what MDEV-9711 is about. >> >> 2. Unicode Collation Algorithm (UCA) based collations >> (e.g. utf8_unicode_nopad_ci) >> >> >> 3. The rest (not covered by the previous steps) >> This will include latin1_general_nopad_cs, latin2_czech_nopad_cs >> >> >> This is why asked in the previous letter if utf8_unicode_nopad_ci would >> work for you. >> Also, which MariaDB version do you need this for? Will 10.1 work? >> > > Thank you very much for this explanation. > > Regarding the version: Our O/S usually is Debian which generally uses software which is quite old. This means that I will have to compile MariaDB myself in every case. > The current project where this new collations are needed will probably take about three months before being finished. It would be very nice if we could have the new collations until then. > In the meantime, we'll continue development with MySQL and switch to MariaDB as soon as the new collations are available. Excellent. The fact that you compile from sources makes the things easier. I guess we can add new collations officially into 10.2. Then after 3 month if you'll feel that 10.2 is not mature enough for you, we can make a custom non-official 10.1 tree with the new collations back-ported (it should be trivial to do). > > ci collations would be sufficient as a starting point. Nevertheless, I think there must be a nopad collation for every collation which already exists. > Personally, since we are using utf8 (and no other encoding) in every part of every project, I would prefer to have *all utf8* nopad collations before >having any other nopad collation. But that's just my personal opinion. >We will be grateful for any progress regardless of the order of things being implemented. Thanks. Then we're flexible to implement #2 first, and wait for #1 to be done under terms of Google Summer Of Code 2016. > > Binarus > > _______________________________________________ > Mailing list: https://launchpad.net/~maria-discuss > Post to : maria-discuss(a)lists.launchpad.net > Unsubscribe : https://launchpad.net/~maria-discuss > More help : https://help.launchpad.net/ListHelp >

1 0

Re: [Maria-developers] [Maria-discuss] Collations, trailing spaces and unique indexes
by Binarus 17 Mar '16

17 Mar '16

Alexander, I'd like to continue our conversation from Maria-discuss. >> - I have found my_strnxfrm_unicode in the same file and will need more time to make my opinion of how difficult it will be >> (I don't know what a weight is, so I currently try to understand what the function does at all). > > This function is used to create sort keys for non-indexed ORDER BY, > for these cases: > > - ORDER BY on an expression > - ORDER BY on a column that does not have an index > > > The idea is exactly the same with the C function strxfrm. > See "man strxfrm". > > The code implements non-indexed ORDER BY in filesort.cc > as follows: > > 1. It calls *_strnxfrm_* functions for all records and converts > CHAR/VARCHAR/TEXT values into their fixed length binary sortable keys. > > 2 . Then executes binary sorting on these keys. > > By the way, fixing this function might be tricky. > > > Currently my_strnxfrm_unicode() pads the tail using weights of the SPACE > character. > The NO PAD version will need to pad the tail using a weight which is > less than the weight of the smallest possible character. > > This should be easy for UCA bases collations (e.g. > utf8_unicode_nopad_ci), because the smallest possible > character in UCA collations is "U+0009 HORIZONTAL TABULATION", > and its weight is 0x0201. So we can just pad the sort key > using a smaller value 0x0200. > > > But I'm not sure yet what to do with 8-bit collations, > which usually use 0x00 as weight for the smallest character. > So we don't have a smaller value. > There are two options here: > > 1. Pad with 0x00. But this will mean that 'aaa<min>' and just 'aaa' > will have unpredictable order when doing ORDER BY without an index > (where <min> is the smallest possible character in the collation). > > As the smallest character in non-UCA collations is usually > "U+0000 NULL", this will mean that 'aaa\0' and just 'aaa' > will have unpredictable order. > > 2. Reserve extra bytes at the end of the key, to store the true length, so > - 'aaa\0' will have the key '4141410004' > - 'aaa' will have the key '4141410003', and therefore will always > be sorted before 'aaa\0'. > > I'm inclined towards #2, to have consistent ORDER BY behavior > with and without indexes. Wow, thanks for explaining. I now have learned that I probably can't assist you in solving that problem (I haven't been in unicode, weights, sorting and character sets very deep until now). But if it helps, you eventually could break the problem into two steps: 1) Fix the problem with unique indexes 2) Fix ordering for any possible situation The problems we have encountered in real life so far are all related to 1), i.e. we would be happy if unique indexes would consider 'a' and 'a ' being different values. In contrast, ordering is not that important for us. I don't know what others are thinking about that, though. >> >> - My main problem: I did not find my_strnncollsp_utf8_general_ci anywhere (nor in the same neither in any other file). Where is it? > > The function name is just "my_strnncollsp_utf8". OK, thank you very much. I must have misunderstood something. >> Furthermore, studying the code has led to some questions; for example, there already seems to be a #define which controls the padding-when-comparing mode, but only for the _cs collations? > > Can you please clarify which lines do you mean? I am referring to the file ctype-utf8.c, line 5698. There is: #ifndef VARCHAR_WITH_DIFF_ENDSPACE_ARE_DIFFERENT_FOR_UNIQUE diff_if_only_endspace_difference= 0; #endif I haven't been able yet to thoroughly analyze where and how this is used, but at least somebody obviously already had some thoughts whether or not trailing spaces should make a difference in UNIQUEs :-) Regards, Binarus

2 1

Re: [Maria-developers] bf2eaa7: MDEV-9382: After updating mariadb server apt-configure fails
by Sergei Golubchik 16 Mar '16

16 Mar '16

Hi, Nirbhay! On Mar 02, Nirbhay Choubey wrote: > revision-id: bf2eaa7c1548f07ce5de13a406ff9183f082b8b3 (mariadb-10.1.12-4-gbf2eaa7) > parent(s): fd741c1a12e78035dcef9a5684b530fe09cbfdde > author: Nirbhay Choubey > committer: Nirbhay Choubey > timestamp: 2016-03-02 09:41:03 -0500 > message: > > MDEV-9382: After updating mariadb server apt-configure fails > > When acting as a Galera receiver node, server startup may take > more than 30 secs (the current default) as it has to wait for > SST/IST operation to complete besides spending some time doing > wsrep recovery. > > In specific case of this MDEV, the server startup timedout as > it took ~48 secs for the node to fully start. > > A viable workaround would be to adjust the MYSQLD_STARTUP_TIMEOUT > env to a higher value, but raising the default to 60 secs should > fix scenarios like these. > > --- > debian/mariadb-server-10.1.mysql.init | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/debian/mariadb-server-10.1.mysql.init b/debian/mariadb-server-10.1.mysql.init > index 9e098b4..005d0e8 100644 > --- a/debian/mariadb-server-10.1.mysql.init > +++ b/debian/mariadb-server-10.1.mysql.init > @@ -109,7 +109,7 @@ case "${1:-''}" in > /usr/bin/mysqld_safe "${@:2}" > /dev/null 2>&1 & > > # 6s was reported in #352070 to be too little > - for i in $(seq 1 "${MYSQLD_STARTUP_TIMEOUT:-30}"); do > + for i in $(seq 1 "${MYSQLD_STARTUP_TIMEOUT:-60}"); do > sleep 1 This doesn't fix much. Next time somebody will need 61 second and you'll adjust to 120? SST might take many minutes and it's not a good idea to have every user wait this long. Instead you should do what MDEV-8509 says. Add a line like [ -r /etc/default/mariadb ] && . /etc/default/mariadb then one can have a very personal setting of MYSQLD_STARTUP_TIMEOUT in /etc/default/mariadb Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

3 3

[Maria-developers] [GSoC 2016] NO PAD Collations
by Даниил Медведев 16 Mar '16

16 Mar '16

Hi, I am Daniil Medvedev from Lomonosov Moscow State University, a 3rd year student, pursuing my bachelor’s degree in Computer Science. I was interested in project idea NO PAD Collations, and I wish to contribute to MariaDB using my skills and ideas. Technologies that i know: C/C++, git, linux, shell, cmake, some databases. My experience: Some projects on github: https://github.com/medvdanil And I contributed one open source NoSQL DBMS on github last summer. Also I am ready to perform some opening task or do commit in your repo, if it helps me to join GSoC. Thank you, Daniil Medvedev

3 2

[Maria-developers] Gsoc 2016 Mdev 371 Unique index for blob
by Sachin Setia 16 Mar '16

16 Mar '16

Dear Developers, I was doing prototype for this project as Mr Sergei Golubchik suggested "This task is about creating MI_UNIQUEDEF "uniques" instead of MI_KEYDEF "keys" for long unique constraints." Here is my output MariaDB [sachin]> create table tbl(int_key int primary key , blob_key blob unique); Query OK, 0 rows affected (0.03 sec) MariaDB [sachin]> insert into tbl values(1,1); Query OK, 1 row affected (0.01 sec) MariaDB [sachin]> insert into tbl values(2,1); ERROR 1062 (23000): Duplicate entry '1' for key 'blob_key' MariaDB [sachin]> insert into tbl values(2,2); Query OK, 1 row affected (0.00 sec) MariaDB [sachin]> insert into tbl values(3,3); Query OK, 1 row affected (0.00 sec) MariaDB [sachin]> select * from tbl; +---------+----------+ | int_key | blob_key | +---------+----------+ | 1 | 1 | | 2 | 2 | | 3 | 3 | +---------+----------+ 3 rows in set (0.00 sec) I am assuming that unique blob will have 0 zero key . Should i mail you the source code.Please let me know Regards sachin

2 4

[Maria-developers] Introduction - GSOF16
by Chandu Maddila 16 Mar '16

16 Mar '16

Hi Team, I'm Chandra Sekhar Maddila from India and currently masters student of BITS-Pilani. I'm pretty interested in working with MariaDB team as part of GSOF. Some of the items that I'm interested in are MXS-1, MDEV-8100, MDEV-5662, MDEV-371 Here's a quick summary of my work experience and Skill set. - Currently a student of BITS-Pilani <http://www.bits-pilani.ac.in/> (Masters, Computer Science). - Total experience of 6 years in IT industry, as a software developer (My last stint was in *Microsoft Research *[MSR] - India as a research SDE). - Have been working with designing and development of RDBMS systems : With Oracle earlier and SQL Server after joining Microsoft - Involved in various projects like :* Implementing column store indexes in SQL Server, Designing a cloud based monitoring systems for database performance monitoring, tools to migrate data from Oracle to SQL Server, Implementing HA Solution (AlwaysOn) for SQL Server etc...* - Languages/Frameworks: - Asp.Net, C#, Jquery, Javascript, HTML5, CSS, Bootstrap, Knockout, SOAP, REST, JSON, Web Services - WCF, MVVM, XAML, Windows App Development, Azure Cloud services, Azure web services - Java, J2EE, UNIX Shell scripting I worked extensively on both *UNIX and Windows platforms*. *Database domain expertise:* *Oracle* + In depth understanding of Oracle database internals and architecture (Memory management, storage engine and optimizer) + *Performance tuning*: Instance and SQL level tuning with the help of *AWR, Statspack, TKPROF* reports and Explain plans and tools like *OEM* and *TOAD* + Oracle *11g, 10g, 9i* Database Setup, Administration, Monitoring on various flavors of UNIX: RHEL, AIX, SUN Solaris, HPUX – Itanium chipset; Good understanding of Oracle internals and Oracle system architecture + Oracle *RAC, ASM and Grid infrastructure* setup, maintenance and administration; Non RAC to RAC, Non-ASM to ASM conversions; Familiarity with OS level clustering techniques like *HACMP* + All kinds of Database *Backup and Recovery* methodologies: *RMAN *(Hot, Cold, Active Database Duplication)*, Datapump, SQL Loader* etc…; Familiarity with 3rd party backup tools like *CA ArcServe* *SQL Server* + Working experience with SQL 2014, 2012, 2008R2, 2008 and 2005 on different versions of Windows + Very good at *troubleshooting complex issues and working on outages/showstoppers*. + Very good in *database performance and query tuning*: Analyzing *execution plans* and rewriting queries + Administering SQL Server High Availability and Disaster Recovery technologies like *AlwaysOn, Failover Clustering *and *Database mirroring*. + Proficiency and expertise in all varieties of *SQL Server Replication*: *Snapshot Replication, Transactional replication, Merge replication and Change tracking* + Expertise in handling all sorts of *backup technologies: Native, VSS, VDI, Backup to URL etc…* + Expert in handling *database corruption cases and recovering* the databases + Expertise in T-SQL writing stored procedures, triggers and functions. For any queries, feel free to reach out to me @ chandu.maddila(a)gmail.com Thanks, Chandra

2 1

Re: [Maria-developers] Gsoc 2016 Mdev 371 Unique index for blob
by Sergei Golubchik 15 Mar '16

15 Mar '16

Hi, Sachin! First: please send questions like this to maria-developers(a)lists.launchpad.net it's a public mailing list dedicated to MySQL and MariaDB internals, source code, and related things. I am subscribed, so I'll see you mail there, and you may be sure I will, because it won't be accidentally catched by my spam filter, or sorted out in some obscure folder. Furthermore other subscribers will see your question and could reply if I will be not available (e.g. I could be travelling). Thank you. On Mar 14, Sachin Setia wrote: > Thank you sir for your reply but my question is what this pack flag will do > and basically what is pack record will you please suggest source file in > which i can look into to get more understanding of pack length ,pack flag > and pack record There isn't much I can answer - your question is too generic. "pack_flag" is a set of flags that is stored per field in the frm file and it describes some aspects of the field. Search for "pack_flag" in the source tree. If you ask a specific question, I'll be able to provide a more detailed answer. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] [Commits] a0c06ba: Preliminary implementation for the aggregate sum function as a window function
by Sergey Petrunia 14 Mar '16

14 Mar '16

Hi Vicentiu, I think it add_helper() is a really poor choice of name when the function can do removal. Maybe add_or_remove() name would be better? On Mon, Mar 14, 2016 at 03:44:48PM +0200, Vicentiu Ciorbaru wrote: > revision-id: a0c06ba1edb54c8c4705189c0455137a85658297 (mariadb-10.1.8-154-ga0c06ba) > parent(s): ce8a0d8e19e7bf1486a2c80ff6b7a30ef35bf99f > author: Vicențiu Ciorbaru > committer: Vicențiu Ciorbaru > timestamp: 2016-03-14 15:42:00 +0200 > message: > > Preliminary implementation for the aggregate sum function as a window function > > This implementation does not deal with the case where removal of > elements from the window frame causes the item to turn to a null value. > > --- > mysql-test/r/win_sum.result | 42 ++++++++++++++++++++++++++++++++++++++++++ > mysql-test/t/win_sum.test | 32 ++++++++++++++++++++++++++++++++ > sql/item_sum.cc | 29 +++++++++++++++++++++++++---- > sql/item_sum.h | 4 ++++ > sql/sql_window.cc | 1 + > 5 files changed, 104 insertions(+), 4 deletions(-) > > diff --git a/mysql-test/r/win_sum.result b/mysql-test/r/win_sum.result > new file mode 100644 > index 0000000..1db6c6e > --- /dev/null > +++ b/mysql-test/r/win_sum.result > @@ -0,0 +1,42 @@ > +create table t1 ( > +pk int primary key, > +a int, > +b int, > +c real > +); > +insert into t1 values > +(101 , 0, 10, 1.1), > +(102 , 0, 10, 2.1), > +(103 , 1, 10, 3.1), > +(104 , 1, 10, 4.1), > +(108 , 2, 10, 5.1), > +(105 , 2, 20, 6.1), > +(106 , 2, 20, 7.1), > +(107 , 2, 20, 8.15), > +(109 , 4, 20, 9.15), > +(110 , 4, 20, 10.15), > +(111 , 5, NULL, 11.15), > +(112 , 5, 1, 12.25), > +(113 , 5, NULL, 13.35), > +(114 , 5, NULL, 14.50), > +(115 , 5, NULL, 15.65); > +select pk, a, b, sum(b) over (partition by a order by pk ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING), > +sum(c) over (partition by a order by pk ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING) > +from t1; > +pk a b sum(b) over (partition by a order by pk ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING) sum(c) over (partition by a order by pk ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING) > +101 0 10 20 3.2 > +102 0 10 20 3.2 > +103 1 10 20 7.199999999999999 > +104 1 10 20 7.199999999999999 > +105 2 20 40 13.2 > +106 2 20 60 21.35 > +107 2 20 50 20.35 > +108 2 10 30 13.250000000000002 > +109 4 20 40 19.3 > +110 4 20 40 19.3 > +111 5 NULL 1 23.4 > +112 5 1 1 36.75 > +113 5 NULL 1 40.1 > +114 5 NULL 0 43.5 > +115 5 NULL 0 30.15 > +drop table t1; > diff --git a/mysql-test/t/win_sum.test b/mysql-test/t/win_sum.test > new file mode 100644 > index 0000000..3c12b08 > --- /dev/null > +++ b/mysql-test/t/win_sum.test > @@ -0,0 +1,32 @@ > +create table t1 ( > + pk int primary key, > + a int, > + b int, > + c real > +); > + > + > +insert into t1 values > +(101 , 0, 10, 1.1), > +(102 , 0, 10, 2.1), > +(103 , 1, 10, 3.1), > +(104 , 1, 10, 4.1), > +(108 , 2, 10, 5.1), > +(105 , 2, 20, 6.1), > +(106 , 2, 20, 7.1), > +(107 , 2, 20, 8.15), > +(109 , 4, 20, 9.15), > +(110 , 4, 20, 10.15), > +(111 , 5, NULL, 11.15), > +(112 , 5, 1, 12.25), > +(113 , 5, NULL, 13.35), > +(114 , 5, NULL, 14.50), > +(115 , 5, NULL, 15.65); > + > +--sorted_result > +select pk, a, b, sum(b) over (partition by a order by pk ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING), > + sum(c) over (partition by a order by pk ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING) > + > +from t1; > + > +drop table t1; > diff --git a/sql/item_sum.cc b/sql/item_sum.cc > index c78c206..3f4853a 100644 > --- a/sql/item_sum.cc > +++ b/sql/item_sum.cc > @@ -1318,25 +1318,39 @@ void Item_sum_sum::fix_length_and_dec() > bool Item_sum_sum::add() > { > DBUG_ENTER("Item_sum_sum::add"); > + add_helper(false); > + DBUG_RETURN(0); > +} > + > +void Item_sum_sum::add_helper(bool perform_removal) > +{ > + DBUG_ENTER("Item_sum_sum::add_helper"); > if (Item_sum_sum::result_type() == DECIMAL_RESULT) > { > my_decimal value; > const my_decimal *val= aggr->arg_val_decimal(&value); > if (!aggr->arg_is_null(true)) > { > - my_decimal_add(E_DEC_FATAL_ERROR, dec_buffs + (curr_dec_buff^1), > - val, dec_buffs + curr_dec_buff); > + if (perform_removal) > + my_decimal_sub(E_DEC_FATAL_ERROR, dec_buffs + (curr_dec_buff ^ 1), > + dec_buffs + curr_dec_buff, val); > + else > + my_decimal_add(E_DEC_FATAL_ERROR, dec_buffs + (curr_dec_buff ^ 1), > + val, dec_buffs + curr_dec_buff); > curr_dec_buff^= 1; > null_value= 0; > } > } > else > { > - sum+= aggr->arg_val_real(); > + if (perform_removal) > + sum-= aggr->arg_val_real(); > + else > + sum+= aggr->arg_val_real(); > if (!aggr->arg_is_null(true)) > null_value= 0; > } > - DBUG_RETURN(0); > + DBUG_VOID_RETURN; > } > > > @@ -1386,6 +1400,13 @@ my_decimal *Item_sum_sum::val_decimal(my_decimal *val) > return val_decimal_from_real(val); > } > > +void Item_sum_sum::remove() > +{ > + DBUG_ENTER("Item_sum_sum::remove"); > + add_helper(true); > + DBUG_VOID_RETURN; > +} > + > /** > Aggregate a distinct row from the distinct hash table. > > diff --git a/sql/item_sum.h b/sql/item_sum.h > index f1f5b22..24dfd9b 100644 > --- a/sql/item_sum.h > +++ b/sql/item_sum.h > @@ -773,6 +773,10 @@ class Item_sum_sum :public Item_sum_num, > return has_with_distinct() ? "sum(distinct " : "sum("; > } > Item *copy_or_same(THD* thd); > + void remove(); > + > +private: > + void add_helper(bool perform_removal); > }; > > > diff --git a/sql/sql_window.cc b/sql/sql_window.cc > index df9899e..f0d74b1 100644 > --- a/sql/sql_window.cc > +++ b/sql/sql_window.cc > @@ -1584,6 +1584,7 @@ bool JOIN::process_window_functions(List<Item> *curr_fields_list) > } > case Item_sum::COUNT_FUNC: > case Item_sum::SUM_BIT_FUNC: > + case Item_sum::SUM_FUNC: > { > /* > Frame-aware window function computation. It does one pass, but > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

2 1

[Maria-developers] Gsoc 2016 Mdev 371 Unique index for blob
by Sachin Setia 14 Mar '16

14 Mar '16

Hello Developers My name is sachin , a third year undergrad student in India.I am interested in gsoc 2016 I was debugging the maria db for following query create table c4(a1 int primary key,b1 int unique,c1 blob) engine=myisam; In mysql_prepare_create_table there is a function name prepare_create_field which sets the sql_field pack flag . for a1 field it sets the pack flag =27 . do not quiet get what it means.

2 1

Re: [Maria-developers] c0b1148: MDEV-9659 : AWS KMS encryption plugin
by Sergei Golubchik 12 Mar '16

12 Mar '16

Hi, Wlad! Just a couple of comments: On Mar 11, wlad(a)mariadb.com wrote: > + # Relax AWS C++ SDK unreasonably high requirements for CMake version. Use replace utility (from MariaDB build) > + # to patch their CMakeLists.txt Nice solution. But you might need to add a dependency, to ensure that this external project target is built after replace. Unless cmake can, somehow, derive it automatically (you can test it by building just this specific target in a clean tree). > + MY_DIR *dirp = my_dir(".", MYF(0)); > + if (!dirp) > + { > + sql_print_error("Can't scan current directory"); > + DBUG_RETURN(-1); > + } > + for (unsigned int i=0; i < dirp->number_of_files; i++) > + { > + > + KEY_INFO info; > + if (extract_id_and_version(dirp->dir_entry[i].name, &info.key_id, &info.key_version) == 0) > + { > + key_info_cache[KEY_ID_AND_VERSION(info.key_id, info.key_version)]= info; > + latest_version_cache[info.key_id]= max(info.key_version, latest_version_cache[info.key_id]); > + } > + } > + DBUG_RETURN(0); my_dir() uses malloc(), don't forget my_dirend() to free the memory. Regards, Sergei

2 1

Re: [Maria-developers] 66cd0f2: MDEV-9704: ALTER TABLE does not work from client
by Sergei Golubchik 12 Mar '16

12 Mar '16

Hi, OleksandrByelkin! On Mar 11, OleksandrByelkin wrote: > revision-id: 66cd0f2f02bb45707b38e983186ab8e7aae21da9 (mariadb-10.1.8-156-g66cd0f2) > parent(s): 75d8544c0bc611e82078b9e5f59caef9105e4892 > committer: Oleksandr Byelkin > timestamp: 2016-03-11 16:55:57 +0100 > message: > > MDEV-9704: ALTER TABLE does not work from client > > Do not try to send progress packets if client was not informed about them. > > --- > sql/sql_acl.cc | 2 -- > 1 file changed, 2 deletions(-) > > diff --git a/sql/sql_acl.cc b/sql/sql_acl.cc > index cd44673..3ae25d5 100644 > --- a/sql/sql_acl.cc > +++ b/sql/sql_acl.cc > @@ -11900,8 +11900,6 @@ static ulong parse_client_handshake_packet(MPVIO_EXT *mpvio, > (((ulonglong)uint4korr(net->read_pos + 28)) << 32); > client_capabilities|= ext_client_capabilities; > } > - else if (client_capabilities & CLIENT_PROGRESS_OBSOLETE) > - client_capabilities|= MARIADB_CLIENT_PROGRESS; Okay. And I think that now you don't need CLIENT_PROGRESS_OBSOLETE either. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] MDEV-9659 Encryption plugin that uses AWS Key management service
by Sergei Golubchik 12 Mar '16

12 Mar '16

Hi, Wlad! On Mar 03, wlad(a)mariadb.com wrote: > revision-id: c1ed3c0267da2ece7664e9f876314979114b2186 (mariadb-10.1.11-26-gc1ed3c0) > parent(s): a62db8186e40a62366bc86f49a0195e9b687d86d > author: Vladislav Vaintroub > committer: Vladislav Vaintroub > timestamp: 2016-03-03 21:29:29 +0100 > message: > > MDEV-9659 Encryption plugin that uses AWS Key management service Looks pretty good, thanks! See comment below... > diff --git a/cmake/plugin.cmake b/cmake/plugin.cmake > index c24239c..361cb59 100644 > --- a/cmake/plugin.cmake > +++ b/cmake/plugin.cmake > @@ -74,7 +74,9 @@ MACRO(MYSQL_ADD_PLUGIN) > SET(compat "with${compat}") > ENDIF() > > - IF (compat STREQUAL ".") > + IF (ARG_DISABLED) > + SET(howtobuild NO) > + ELSEIF (compat STREQUAL ".") > SET(howtobuild DYNAMIC) > ELSEIF (compat STREQUAL "with.") > IF (NOT ARG_MODULE_ONLY) > @@ -122,7 +124,7 @@ MACRO(MYSQL_ADD_PLUGIN) > > # Build either static library or module > IF (PLUGIN_${plugin} MATCHES "(STATIC|AUTO|YES)" AND NOT ARG_MODULE_ONLY > - AND NOT ARG_DISABLED AND NOT ARG_CLIENT) > + AND NOT ARG_CLIENT) > > IF(CMAKE_GENERATOR MATCHES "Makefiles|Ninja") > # If there is a shared library from previous shared build, > @@ -178,8 +180,7 @@ MACRO(MYSQL_ADD_PLUGIN) > SET (mysql_optional_plugins ${mysql_optional_plugins} PARENT_SCOPE) > ENDIF() > ELSEIF(PLUGIN_${plugin} MATCHES "(DYNAMIC|AUTO|YES)" > - AND NOT ARG_STATIC_ONLY AND NOT WITHOUT_DYNAMIC_PLUGINS > - AND NOT ARG_DISABLED) > + AND NOT ARG_STATIC_ONLY AND NOT WITHOUT_DYNAMIC_PLUGINS) > > ADD_VERSION_INFO(${target} MODULE SOURCES) > ADD_LIBRARY(${target} MODULE ${SOURCES}) a bit more of the explanation would've been nice. in the changeset comment or - even better - pushing this change separately. you can select what files to include in the commit, you know... > diff --git a/plugin/aws_key_management/CMakeLists.txt b/plugin/aws_key_management/CMakeLists.txt > new file mode 100644 > index 0000000..ec9ee46 > --- /dev/null > +++ b/plugin/aws_key_management/CMakeLists.txt > @@ -0,0 +1,144 @@ > +# We build parts of AWS C++ SDK as CMake external project > +# The restrictions of the SDK (https://github.com/awslabs/aws-sdk-cpp/blob/master/README.md) > +# are > + > +# - OS : Windows,Linux or OSX > +# - C++11 compiler : VS2013+, gcc 4.7+, clang 3.3+ > +# - libcurl development package needs to be present on Unixes > +# > +# Since we're using ExternalProject that reads from git repository > +# we also require GIT to be present on the build machine > + > + > +# Give message why the building this plugin is skipped (only if -DVERBOSE is defined) > +# and and bail out. > +MACRO(SKIP_AWS_PLUGIN msg) > + IF(DEFINED VERBOSE) VERBOSE - is it used somewhere else? Or you've invented it on the spot for this plugin? I understand that it can be used in other cmake files, but is it? > + MESSAGE(STATUS "Skip aws_key_management - ${msg}") > + RETURN() > + ENDIF() > +ENDMACRO() Ok, as discussed on irc, I'd suggest to add here a check whether AWS SDK is already installed. Like, CheckLibraryExists or CheckSymbolExists. And if not - then try to download and compile it (which would require new cmake, git, curl, etc). > +IF(CMAKE_VERSION VERSION_LESS "2.8.8") > + SKIP_AWS_PLUGIN("CMake is too old") > +ENDIF() > + > +FIND_PACKAGE(Git) > +IF(NOT GIT_FOUND) > + SKIP_AWS_PLUGIN("no GIT") > +ENDIF() > + > +INCLUDE(ExternalProject) > +IF (NOT(WIN32 OR APPLE OR (CMAKE_SYSTEM_NAME MATCHES "Linux"))) > + SKIP_AWS_PLUGIN("OS unsupported by AWS SDK") > +ENDIF() > + > +IF(UNIX) > + FIND_PACKAGE(CURL) > + IF(NOT CURL_FOUND) > + SKIP_AWS_PLUGIN("AWS C++ SDK requires libcurl development package") > + ENDIF() > + SET(PIC_FLAG -fPIC) > +ENDIF() > + > +SET(CXX11_FLAGS) > +SET(OLD_COMPILER_MSG "AWS SDK requires c++11 -capable compiler (minimal supported versions are g++ 4.7, clang 3.3, VS2103)") > +IF(CMAKE_CXX_COMPILER_ID MATCHES "GNU") > + EXECUTE_PROCESS(COMMAND ${CMAKE_CXX_COMPILER} -dumpversion OUTPUT_VARIABLE GCC_VERSION) eh? I thought cmake knows compiler versions... let me see This one: CMAKE_<LANG>_COMPILER_VERSION > + IF (GCC_VERSION VERSION_LESS 4.7) > + SKIP_AWS_PLUGIN("${OLD_COMPILER_MSG}") > + ENDIF() > + SET(CXX11_FLAGS "-std=c++11") Instead of checking the compiler version, I would rather suggest to check for a feature that you need to use. Like MY_CHECK_CXX_COMPILER_FLAG(-std=c++11) IF (NOT HAVE_CXX__STD_C__11) SKIP_AWS_PLUGIN("${OLD_COMPILER_MSG}") ENDIF() Something like that. Not sure about HAVE_CXX__STD_C__11 name, though. > +ELSEIF (CMAKE_CXX_COMPILER_ID MATCHES "Clang") > + IF (CMAKE_CXX_COMPILER_VERSION VERSION_LESS 3.3) > + SKIP_AWS_PLUGIN("${OLD_COMPILER_MSG}") > + ENDIF() > + SET(CXX11_FLAGS "-stdlib=libc++") Same. > +ELSEIF(MSVC) > + IF (MSVC_VERSION LESS 1800) > + SKIP_AWS_PLUGIN("${OLD_COMPILER_MSG}") > + ENDIF() > +ELSE() > + SKIP_AWS_PLUGIN("Compiler not supported by AWS C++ SDK") > +ENDIF() > + > +IF(WIN32) > + SET(EXTRA_SDK_CMAKE_FLAGS -DCMAKE_CXX_FLAGS_DEBUGOPT="" -DCMAKE_EXE_LINKER_FLAGS_DEBUGOPT="") For consistency you can put this inside ELSEIF(MSVC), that's where you set options for gcc and clang. > +ENDIF() > + > +FIND_PROGRAM(PERL_EXECUTABLE perl) > + > +IF(PERL_EXECUTABLE) > + # CMake minimimal version for building AWS SDK can be less than 3.1.2 > + # Downgrade this requirement. > + SET(AWS_SDK_PATCH_COMMAND > + PATCH_COMMAND ${PERL_EXECUTABLE} -i.bak -pe s/3.1.2/2.8/g ${CMAKE_BINARY_DIR}/aws-sdk-cpp/CMakeLists.txt) 1. Without perl it'll require 3.1.2, right? 2. Couldn't you do that with cmake alone? 3. Is that safe at all - you pull the latest version from git, they can start using cmake-3.1.2 features any time. > +ENDIF() > + > +ExternalProject_Add( > + aws_sdk_cpp > + GIT_REPOSITORY "https://github.com/awslabs/aws-sdk-cpp.git" > + UPDATE_COMMAND "" > + ${AWS_SDK_PATCH_COMMAND} > + SOURCE_DIR "${CMAKE_BINARY_DIR}/aws-sdk-cpp" > + CMAKE_ARGS > + -DBUILD_ONLY=aws-cpp-sdk-kms -DSTATIC_LINKING=1 > + "-DCMAKE_CXX_FLAGS_DEBUG=${CMAKE_CXX_FLAGS_DEBUG} ${PIC_FLAG}" > + "-DCMAKE_CXX_FLAGS_RELWITHDEBINFO=${CMAKE_CXX_FLAGS_RELWITHDEBINFO} ${PIC_FLAG}" > + "-DCMAKE_CXX_FLAGS_RELEASE=${CMAKE_CXX_FLAGS_RELEASE} ${PIC_FLAG}" > + "-DCMAKE_CXX_FLAGS_MINSIZEREL=${CMAKE_CXX_FLAGS_MINSIZEREL} ${PIC_FLAG}" > + ${EXTRA_SDK_CMAKE_FLAGS} > + -DCMAKE_INSTALL_PREFIX=${CMAKE_BINARY_DIR}/aws_sdk_cpp > + TEST_COMMAND "" > +) > + > +# Figure out where AWS installs SDK libraries > +# The below is defined in AWS SDK's CMakeLists.txt > +# (and their handling is weird, every OS has special install directory) > +if(WIN32) > + SET(SDK_INSTALL_BINARY_PREFIX "windows") > +ELSEIF(APPLE) > + SET(SDK_INSTALL_BINARY_PREFIX "mac") > +ELSEIF(UNIX) > + SET(SDK_INSTALL_BINARY_PREFIX "linux") > +ENDIF() > +IF(NOT APPLE) > + IF(CMAKE_SIZEOF_VOID_P EQUAL 8) > + SET(SDK_INSTALL_BINARY_PREFIX "${SDK_INSTALL_BINARY_PREFIX}/intel64") > + ELSE() > + SET(SDK_INSTALL_BINARY_PREFIX "${SDK_INSTALL_BINARY_PREFIX}/ia32") > + ENDIF() > +ENDIF() > +IF(CMAKE_CONFIGURATION_TYPES) > + SET(SDK_INSTALL_BINARY_PREFIX "${SDK_INSTALL_BINARY_PREFIX}/${CMAKE_CFG_INTDIR}") > +ENDIF() > + > +# We do not need to build the whole SDK , just 2 of its libs > +set(AWS_SDK_LIBS aws-cpp-sdk-core aws-cpp-sdk-kms) > + > +FOREACH(lib ${AWS_SDK_LIBS}) > + ADD_LIBRARY(${lib} STATIC IMPORTED) > + ADD_DEPENDENCIES(${lib} aws_sdk_cpp) > + SET(loc "${CMAKE_BINARY_DIR}/aws_sdk_cpp/lib/${SDK_INSTALL_BINARY_PREFIX}/${CMAKE_STATIC_LIBRARY_PREFIX}${lib}${CMAKE_STATIC_LIBRARY_SUFFIX}") > + SET_TARGET_PROPERTIES(${lib} PROPERTIES IMPORTED_LOCATION ${loc}) > + IF(WIN32) > + SET_TARGET_PROPERTIES(${lib} PROPERTIES IMPORTED_LINK_INTERFACE_LIBRARIES "bcrypt;winhttp;wininet;userenv") > + ELSE() > + SET_TARGET_PROPERTIES(${lib} PROPERTIES IMPORTED_LINK_INTERFACE_LIBRARIES "${SSL_LIBRARIES};${CURL_LIBRARIES}") > + ENDIF() > +ENDFOREACH() > + > +IF(CMAKE_SYSTEM_NAME MATCHES "Linux") > + # Need whole-archive , otherwise static libraries are not linked > + SET(AWS_SDK_LIBS -Wl,--whole-archive ${AWS_SDK_LIBS} -Wl,--no-whole-archive) > +ENDIF() > + > +SET_TARGET_PROPERTIES(aws_sdk_cpp PROPERTIES EXCLUDE_FROM_ALL TRUE) > +INCLUDE_DIRECTORIES(${CMAKE_BINARY_DIR}/aws_sdk_cpp/include) > +SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX11_FLAGS}") > +MYSQL_ADD_PLUGIN(aws_key_management aws_key_management_plugin.cc > + MODULE_ONLY COMPONENT aws-key-management > + LINK_LIBRARIES ${AWS_SDK_LIBS} DISABLED) > diff --git a/plugin/aws_key_management/aws_key_management_plugin.cc b/plugin/aws_key_management/aws_key_management_plugin.cc > new file mode 100644 > index 0000000..fa7d042 > --- /dev/null > +++ b/plugin/aws_key_management/aws_key_management_plugin.cc > @@ -0,0 +1,611 @@ > +/* > + Copyright (c) 2016 MariaDB Corporation > + > + This program is free software; you can redistribute it and/or modify > + it under the terms of the GNU General Public License as published by > + the Free Software Foundation; version 2 of the License. > + > + This program is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + GNU General Public License for more details. > + > + You should have received a copy of the GNU General Public License > + along with this program; if not, write to the Free Software > + Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */ > + > + > +#include <my_global.h> > +#include <my_pthread.h> > +#include <my_sys.h> > +#include <mysql/plugin_encryption.h> > +#include <my_crypt.h> > +#include <string.h> > +#include <stdio.h> > +#include <stdlib.h> > +#include <mysqld_error.h> > +#include <base64.h> > +#include <map> > +#include <algorithm> > +#include <string> > +#include <vector> > +#include <iterator> > +#include <sstream> > +#include <fstream> > +#ifndef _WIN32 > +#include <sys/types.h> > +#include <dirent.h> > +#endif Use cmake checks when there are some, HAVE_DIRENT_H, HAVE_SYS_TYPES_H, HAVE_STRING_H, HAVE_STDLIB_H. Hmm, HAVE_STDLIB_H doesn't make sense to me :) could you remove it from configure.cmake, please? In a separate changeset. May be HAVE_SYS_TYPES_H too, not sure. But do use HAVE_DIRENT_H and HAVE_STDLIB_H above instead of _WIN32. > +#include <aws/core/client/AWSError.h> > +#include <aws/core/utils/logging/AWSLogging.h> > +#include <aws/core/utils/logging/ConsoleLogSystem.h> > +#include <aws/kms/KMSClient.h> > +#include <aws/kms/model/DecryptRequest.h> > +#include <aws/kms/model/DecryptResult.h> > +#include <aws/kms/model/GenerateDataKeyWithoutPlaintextRequest.h> > +#include <aws/kms/model/GenerateDataKeyWithoutPlaintextResult.h> > +#include <aws/core/utils/Outcome.h> > +using namespace std; > +using namespace Aws::KMS; > +using namespace Aws::KMS::Model; > +using namespace Aws::Utils::Logging; > +extern void sql_print_error(const char *format, ...); > +extern void sql_print_warning(const char *format, ...); > +extern void sql_print_information(const char *format, ...); > + > + > +/* Plaintext key info struct */ > +struct KEY_INFO > +{ > + unsigned int key_id; > + unsigned int key_version; > + unsigned int length; > + unsigned char data[MY_AES_MAX_KEY_LENGTH]; > + bool load_failed; /* if true, do not attempt to reload?*/ > +public: > + KEY_INFO() : key_id(0), key_version(0), length(0), load_failed(false){}; > +}; > +#define KEY_ID_AND_VERSION(key_id,version) ((longlong)key_id << 32 | version) > + > +/* Cache for the latest version, per key id */ > +static std::map<uint, uint> latest_version_cache; > + > +/* Cache for plaintext keys */ > +static std::map<ulonglong, KEY_INFO> key_info_cache; > + > +static const char *key_spec_names[]={ "AES_128", "AES_256", 0 }; > + > +/* Plugin variables */ > +static char* master_key_id; > +static unsigned long key_spec; > +static unsigned long log_level; > +static int rotate_key; > + > +/* AWS functionality*/ > +static int aws_decrypt_key(const char *path, KEY_INFO *info); > +static int aws_generate_datakey(uint key_id, uint version); > + > +static int extract_id_and_version(const char *name, uint *id, uint *ver); > +static unsigned int get_latest_key_version(unsigned int key_id); > +static unsigned int get_latest_key_version_nolock(unsigned int key_id); > +static int load_key(KEY_INFO *info); > + > +/* Mutex to serialize access to caches */ > +static pthread_mutex_t mtx; Use PSI, please. That's mysql_mutex_t, etc. > + > +static Aws::KMS::KMSClient *client; > + > +/* Redirect AWS trace to error log */ > +class MySQLLogSystem : public Aws::Utils::Logging::FormattedLogSystem > +{ > +public: > + > + using Base = FormattedLogSystem; > + MySQLLogSystem(LogLevel logLevel) : > + Base(logLevel) > + { > + } > + virtual LogLevel GetLogLevel(void) const override > + { > + return (LogLevel)log_level; > + } > + virtual ~MySQLLogSystem() > + { > + } > + > +protected: > + virtual void ProcessFormattedStatement(Aws::String&& statement) override > + { > +#ifdef _WIN32 > + /* > + On Windows, we can't use C runtime functions to write to stdout, > + because we compile with static C runtime, so plugin has a stdout > + different from server. Thus we're using WriteFile(). > + */ > + DWORD nSize= (DWORD)statement.size(); > + DWORD nWritten; > + const char *s= statement.c_str(); > + HANDLE h= GetStdHandle(STD_OUTPUT_HANDLE); > + > + WriteFile(h, s, nSize, &nWritten, NULL); > +#else > + printf("%s", statement.c_str()); printf? You have declared sql_print_information above, why wouldn't you use it here? > +#endif > + } > +}; > + > +/* > + Read filenames from the current directory. > + This is used in plugin initilization. We check for > + plugin specific names aws-kms-key.<id>.<version> > + to find out what keys/key versions are already there. > +*/ > +static vector<string> read_dir() > +{ > + vector<string> v; > + DBUG_ENTER("read_dir"); > +#ifdef _WIN32 > + WIN32_FIND_DATA ffd; > + HANDLE h= FindFirstFile("*", &ffd); > + if (h == INVALID_HANDLE_VALUE) > + { > + DBUG_RETURN(v); > + } > + do > + { > + /* handle file possibly updates caches*/ > + v.push_back(ffd.cFileName); > + } while (FindNextFile(h, &ffd)); > + FindClose(h); > +#else > + DIR *dir= opendir("."); > + if (!dir) > + { > + DBUG_RETURN(v); > + } > + struct dirent *dp; > + while ((dp= readdir(dir)) != NULL) > + { > + v.push_back(dp->d_name); > + } > + closedir(dir); > +#endif > + DBUG_RETURN(v); ok. although you could've used my_dir and save yourself some troubles :) > +} > + > + > +/* > + Plugin initialization. > + > + Create KMS client and scan datadir to find out which keys and versions > + are present. > +*/ > +static int plugin_init(void *p) > +{ > + DBUG_ENTER("plugin_init"); > + client = new KMSClient(); > + if (!client) > + { > + sql_print_error("Can not initialize KMS client"); > + DBUG_RETURN(-1); > + } > + InitializeAWSLogging(Aws::MakeShared<MySQLLogSystem>("aws_key_management_plugin", (Aws::Utils::Logging::LogLevel) log_level)); > + pthread_mutex_init(&mtx, NULL); > + vector<string> filenames= read_dir(); > + for (size_t i= 0; i < filenames.size(); i++) > + { > + KEY_INFO info; > + if (extract_id_and_version(filenames[i].c_str(), &info.key_id, &info.key_version) == 0) > + { > + key_info_cache[KEY_ID_AND_VERSION(info.key_id, info.key_version)]= info; > + latest_version_cache[info.key_id]= max(info.key_version, latest_version_cache[info.key_id]); > + } > + } > + DBUG_RETURN(0); > +} > + > + > + > +static int plugin_deinit(void *p) > +{ > + DBUG_ENTER("plugin_deinit"); > + latest_version_cache.clear(); > + key_info_cache.clear(); > + pthread_mutex_destroy(&mtx); > + delete client; > + ShutdownAWSLogging(); > + DBUG_RETURN(0); > +} > + > +/* Generate filename to store the ciphered key */ > +static void format_keyfile_name(char *buf, size_t size, uint key_id, uint version) > +{ > + snprintf(buf, size, "aws-kms-key.%u.%u", key_id, version); > +} > + > +/* Extract key id and version from file name */ > +static int extract_id_and_version(const char *name, uint *id, uint *ver) > +{ > + int len; > + int n= sscanf(name, "aws-kms-key.%u.%u%n", id, ver, &len); > + if (n == 2 && *id > 0 && *ver > 0 && len == (int)strlen(name)) > + return 0; > + return 1; > +} > + > +/* > + Decrypt key stored in aws-kms-key.<id>.<version> > + Cache the decrypted key. > +*/ > +static int load_key(KEY_INFO *info) > +{ > + int ret; > + char path[256]; > + DBUG_ENTER("load_key"); > + DBUG_PRINT("enter", ("id=%u,ver=%u", info->key_id, info->key_version)); > + format_keyfile_name(path, sizeof(path), info->key_id, info->key_version); > + ret= aws_decrypt_key(path, info); > + if (ret) > + info->load_failed= true; > + > + latest_version_cache[info->key_id]= max(latest_version_cache[info->key_id], info->key_version); > + key_info_cache[KEY_ID_AND_VERSION(info->key_id, info->key_version)]= *info; > + > + if (!ret) > + { > + sql_print_information("AWS KMS plugin: loaded key %u, version %u, key length %u bit", > + info->key_id, info->key_version,(uint)info->length*8); > + } > + else > + { > + sql_print_warning("AWS KMS plugin: key %u, version %u could not be decrypted", > + info->key_id, info->key_version); > + } > + DBUG_RETURN(ret); > +} > + > + > +/* > + Get latest version for the key. > + > + If key is not decrypted yet, this function also decrypt the key > + and error will be returned if decryption fails. > + > + The reason for that is that InnoDB crashes > + in case errors are returned by get_key(), Shall we rather fix that in InnoDB? Do you remember where it crashes? > + > + A new key will be created if it does not exist, provided there is > + valid master_key_id. > +*/ > +static unsigned int get_latest_key_version(unsigned int key_id) > +{ > + unsigned int ret; > + DBUG_ENTER("get_latest_key_version"); > + > + pthread_mutex_lock(&mtx); > + ret= get_latest_key_version_nolock(key_id); why do you need get_latest_key_version_nolock if you never use it directly? > + pthread_mutex_unlock(&mtx); > + > + DBUG_PRINT("info", ("key=%u,ret=%u", key_id, ret)); > + DBUG_RETURN(ret); > +} > + > +static unsigned int get_latest_key_version_nolock(unsigned int key_id) > +{ > + KEY_INFO info; > + uint ver; > + DBUG_ENTER("get_latest_key_version_nolock"); > + ver= latest_version_cache[key_id]; > + if (ver > 0) > + { > + info= key_info_cache[KEY_ID_AND_VERSION(key_id, ver)]; > + } > + if (info.load_failed) > + { > + /* Decryption failed previously, don't retry */ > + DBUG_RETURN(ENCRYPTION_KEY_VERSION_INVALID); > + } > + else if (ver > 0) > + { > + /* Key encrypted already, return it*/ may be s/encrypted already/exists/ ? > + if (info.length > 0) > + DBUG_RETURN(ver); > + } > + else // (ver == 0) > + { > + /* Generate a new key, version 1 */ > + if (!master_key_id[0]) > + { > + my_printf_error(ER_UNKNOWN_ERROR, > + "Can't generate key %u, because aws_key_management_master_key_id parameter is not set", > + MYF(ME_JUST_WARNING), key_id); > + DBUG_RETURN(ENCRYPTION_KEY_VERSION_INVALID); > + } > + if (aws_generate_datakey(key_id, 1) != 0) > + DBUG_RETURN(ENCRYPTION_KEY_VERSION_INVALID); > + info.key_id= key_id; > + info.key_version= 1; > + info.length= 0; > + } > + > + if (load_key(&info)) > + DBUG_RETURN(ENCRYPTION_KEY_VERSION_INVALID); > + DBUG_RETURN(info.key_version); > +} > + > + > +/* > + Decrypt a file with KMS > +*/ > +static int aws_decrypt_key(const char *path, KEY_INFO *info) > +{ > + DBUG_ENTER("aws_decrypt_key"); > + > + /* Read file content into memory */ > + ifstream ifs(path, ios::binary | ios::ate); > + if (!ifs.good()) > + { > + sql_print_error("can't open file %s", path); > + } > + size_t pos = (size_t)ifs.tellg(); > + std::vector<char> contents(pos); > + ifs.seekg(0, ios::beg); > + ifs.read(&contents[0], pos); > + > + /* Decrypt data the with AWS */ > + DecryptRequest request; > + Aws::Utils::ByteBuffer byteBuffer((unsigned char *)contents.data(), pos); > + request.SetCiphertextBlob(byteBuffer); > + DecryptOutcome outcome = client->Decrypt(request); > + if (!outcome.IsSuccess()) > + { > + sql_print_error("AWS KMS plugin: Decrypt failed for %s : %s", path, > + outcome.GetError().GetMessage().c_str()); > + DBUG_RETURN(-1); > + } > + Aws::Utils::ByteBuffer plaintext = outcome.GetResult().GetPlaintext(); > + size_t len = plaintext.GetLength(); > + > + if (len > (int)sizeof(info->data)) > + { > + sql_print_error("AWS KMS plugin: encoding key too large for %s", path); > + DBUG_RETURN(ENCRYPTION_KEY_BUFFER_TOO_SMALL); > + } > + memcpy(info->data, plaintext.GetUnderlyingData(), len); > + info->length= len; > + DBUG_RETURN(0); > +} > + > + > +/* Generate a new datakey and store it a file */ > +static int aws_generate_datakey(uint keyid, uint version) > +{ > + > + DBUG_ENTER("aws_generate_datakey"); > + GenerateDataKeyWithoutPlaintextRequest request; > + request.SetKeyId(master_key_id); > + request.SetKeySpec(DataKeySpecMapper::GetDataKeySpecForName(key_spec_names[key_spec])); > + > + GenerateDataKeyWithoutPlaintextOutcome outcome; > + outcome= client->GenerateDataKeyWithoutPlaintext(request); > + if (!outcome.IsSuccess()) > + { > + sql_print_error("AWS KMS plugin : GenerateDataKeyWithoutPlaintext failed : %s - %s", > + outcome.GetError().GetExceptionName().c_str(), > + outcome.GetError().GetMessage().c_str()); > + DBUG_RETURN(-1); > + } > + > + string out; > + char filename[20]; > + Aws::Utils::ByteBuffer byteBuffer = outcome.GetResult().GetCiphertextBlob(); > + > + format_keyfile_name(filename, sizeof(filename), keyid, version); > + int fd= my_open(filename, O_RDWR | O_CREAT, 0); > + if (fd < 0) > + { > + sql_print_error("AWS KMS plugin: Can't create file %s", filename); > + DBUG_RETURN(-1); > + } > + size_t len= byteBuffer.GetLength(); > + if (my_write(fd, byteBuffer.GetUnderlyingData(), len, 0) != len) > + { > + sql_print_error("AWS KMS plugin: can't write to %s", filename); > + my_close(fd, 0); > + my_delete(filename, 0); > + DBUG_RETURN(-1); > + } > + my_close(fd, 0); > + sql_print_information("AWS KMS plugin: generated encrypted datakey for key id=%u, version=%u", > + keyid, version); > + DBUG_RETURN(0); > +} > + > +/* Key rotation for a single key */ > +static int rotate_single_key(uint key_id) > +{ > + uint ver; > + ver= latest_version_cache[key_id]; > + > + if (!ver) > + { > + my_printf_error(ER_UNKNOWN_ERROR, "key %u does not exist", MYF(ME_JUST_WARNING), key_id); > + return -1; > + } > + else if (aws_generate_datakey(key_id, ver + 1)) > + { > + my_printf_error(ER_UNKNOWN_ERROR, "Could not generate datakey for key id= %u, ver= %u", > + MYF(ME_JUST_WARNING), key_id, ver); > + return -1; > + } > + else > + { > + KEY_INFO info; > + info.key_id= key_id; > + info.key_version = ver + 1; > + if (load_key(&info)) > + { > + my_printf_error(ER_UNKNOWN_ERROR, "Could not load datakey for key id= %u, ver= %u", > + MYF(ME_JUST_WARNING), key_id, ver); > + return -1; > + } > + } > + return 0; > +} > + > +/* Key rotation for all key ids */ > +static int rotate_all_keys() > +{ > + int ret= 0; > + for (map<uint, uint>::iterator it= latest_version_cache.begin(); it != latest_version_cache.end(); it++) > + { > + ret= rotate_single_key(it->first); > + if (ret) > + break; > + } > + return ret; > +} > + > +static void update_rotate(MYSQL_THD, struct st_mysql_sys_var *, void *, const void *val) > +{ > + if (!master_key_id[0]) > + { > + my_printf_error(ER_UNKNOWN_ERROR, > + "aws_key_management_master_key_id must be set to generate new data keys", MYF(ME_JUST_WARNING)); > + return; > + } > + pthread_mutex_lock(&mtx); > + rotate_key= *(int *)val; > + switch (rotate_key) > + { > + case 0: > + break; > + case -1: > + rotate_all_keys(); > + break; > + default: > + rotate_single_key(rotate_key); > + break; > + } > + rotate_key= 0; > + pthread_mutex_unlock(&mtx); > +} > + > +static unsigned int get_key( > + unsigned int key_id, > + unsigned int version, > + unsigned char* dstbuf, > + unsigned int* buflen) > +{ > + KEY_INFO info; > + > + DBUG_ENTER("get_key"); > + pthread_mutex_lock(&mtx); > + info= key_info_cache[KEY_ID_AND_VERSION(key_id, version)]; > + if (info.length == 0 && !info.load_failed) > + { > + info.key_id= key_id; > + info.key_version= version; > + load_key(&info); > + } > + pthread_mutex_unlock(&mtx); > + > + if (info.load_failed) > + DBUG_RETURN(ENCRYPTION_KEY_VERSION_INVALID); > + if (*buflen < info.length) > + { > + *buflen= info.length; > + DBUG_RETURN(ENCRYPTION_KEY_BUFFER_TOO_SMALL); > + } > + *buflen= info.length; > + memcpy(dstbuf, info.data, info.length); > + DBUG_RETURN(0); > +} > + > + > +/* Plugin defs */ > +struct st_mariadb_encryption aws_key_management_plugin= { > + MariaDB_ENCRYPTION_INTERFACE_VERSION, > + get_latest_key_version, > + get_key, > + // use default encrypt/decrypt functions > + 0, 0, 0, 0, 0 > +}; > + > + > +static TYPELIB key_spec_typelib = > +{ > + array_elements(key_spec_names) - 1, "", > + key_spec_names, NULL > +}; > + > +const char *log_level_names[] = > +{ > + "Off", > + "Fatal", > + "Error", > + "Warn", > + "Info", > + "Debug", > + "Trace", > + 0 > +}; > + > +static TYPELIB log_level_typelib = > +{ > + array_elements(log_level_names) - 1, "", > + log_level_names, NULL > +}; > + > +static MYSQL_SYSVAR_STR(master_key_id, master_key_id, > + PLUGIN_VAR_RQCMDARG | PLUGIN_VAR_MEMALLOC, > + "Key id for master encryption key. Used to create new datakeys. If not set, no new keys will be created", > + NULL, NULL, ""); > + > +static MYSQL_SYSVAR_ENUM(key_spec, key_spec, > + PLUGIN_VAR_RQCMDARG, > + "Encryption algorithm used to create new keys.", > + NULL, NULL, 0, &key_spec_typelib); > + > + > +static MYSQL_SYSVAR_ENUM(log_level, log_level, > + PLUGIN_VAR_RQCMDARG, > + "Logging for AWS API", > + NULL, NULL, 0, &log_level_typelib); > + > + > +static MYSQL_SYSVAR_INT(rotate_key, rotate_key, > + PLUGIN_VAR_RQCMDARG, > + "Set this variable to key id to perform rotation of the key. Specify -1 to rotate all keys", > + NULL, update_rotate, 0, -1, INT_MAX, 1); > + > +static struct st_mysql_sys_var* settings[]= { > + MYSQL_SYSVAR(master_key_id), > + MYSQL_SYSVAR(key_spec), > + MYSQL_SYSVAR(rotate_key), > + MYSQL_SYSVAR(log_level), > + NULL > +}; > + > +/* > + Plugin library descriptor > +*/ > +maria_declare_plugin(aws_key_management) > +{ > + MariaDB_ENCRYPTION_PLUGIN, > + &aws_key_management_plugin, > + "aws_key_management", > + "MariaDB Corporation", > + "AWS key management plugin", > + PLUGIN_LICENSE_GPL, > + plugin_init, > + plugin_deinit, > + 0x0100, > + NULL, > + settings, > + "1.0", > + MariaDB_PLUGIN_MATURITY_EXPERIMENTAL > +} > +maria_declare_plugin_end; Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 2

[Maria-developers] GSoC 2016 MHA GTID support.
by Charles Dirk 11 Mar '16

11 Mar '16

Hi, I am a computer science student at the university of Cape Town and wish to participate in GSoC 2016 with MariaDB Foundation. I am interested if working on making MHA understand GTIDs. I am proficient in C/C+, Python/Perl and Im equally familiar with replication. I look forward to hearing from mentors soon. Cheers, Charles.

3 3

[Maria-developers] MDEV-9709 Unexpected modification of value and warning about out of range value upon ALTER
by Alexander Barkov 11 Mar '16

11 Mar '16

Hi Elena, can you please review a patch for MDEV-9709? I'm quite sure about the code change, it's trivial. Please have a look into the tests. Thanks.

1 0

[Maria-developers] GSoC 2016 Maxscale Filter
by Lisa Brinson 08 Mar '16

08 Mar '16

Hi everyone, My name is Lisa Brinson and I am a junior Computer science student at Oregon State University. I wish to participate in an open source project through GSoC 2016. I am proficient in C and C++ as well as git. I have looked through the proposed projects and I have developed interest in the Maxscale tasks especially the filters. I have been going through the filter API and have also looked at some sample filters in server/modules/filter directory. I will try to write my first filter(a hello world filter) to get familiar with the API. I will be submitting 3 proposals for the 3 Maxscale filters hoping one will be selected. Best regards, Lisa.

3 4

[Maria-developers] Questions regarding GSoC 2016 and Import / Export
by Dan Ungureanu 08 Mar '16

08 Mar '16

Hello, I am a second year student currently attending University Politehnica of Bucharest and I am interested in contributing to MariaDB during this year's Google Summer of Code. I have looked at the ideas you published in the Knowledge Base [1] and two projects caught my attention. The first one is related to the import and export process related to dynamic columns [2] and the second one is related to the GIS enhancements [3]. I am willing to implement both of them, but I am not sure whether the time will allow me properly implement both of them. I cannot estimate how much effort do these projects require as I am still trying to become acquainted with the code base. I hope I am not underestimating the projects and some guidance from one of the mentors (Oleksandr Byelkin, Georg Richter or Alexey Botchkov - Holyfoot) is certainly welcome. I will focus on "Import and export popular data formats from and to dynamic columns" and as soon as I am done with this one, hopefully I will be able to work on GIS enhancements as well. Before continuing, I would like to clarify some aspects related to these projects. Import and export popular data formats from and to dynamic columns: - Can be split in four main tasks: a) adding support for arrays (vectors) in dynamic columns. Currently, dynamic columns only store dictionaries (maps); b) implement a library that can import (parse) and export (build in a specific format) data to and from dynamic columns; c) provide interface to access to the new import and export mechanism; d) provide unit tests for the new features. - The second part (b) of the project is mostly standalone and requires little edits to the project and the third part (c) will be mostly shared by the server and the C connector. - The outcome of (b) is going to be a minimal parsing and building library for various formats. - Some of the formats that I plan to support are: JSON, XML, CSV (maybe XLS too?), PHP Array, YAML, SQL (export as INSERT INTO statement?), etc. One of the key aspects I have to keep in mind when designing the library is to provide a extensible interface to quickly implement new formats in the future. - Probably, most of my work will be located in: - ./server/sql/item_*.(cc|h) - implement user defined functions - ./server/mysys/ma_dyncol.c - provide interface for user defined functions - ./server/include/ma_dyncol.h - ./mariadb-connector-c/libmariadb/mariadb_dyncol.c - provide interface in C Connector - ./mariadb-connector-c/include/mariadb_dyncol.h GIS enhancements: - This project is based on extending the current functionality related to GIS. I am not very familiar with GIS and I still have to do more research based on documents [4], [5] and [6], but the biggest part of the project consists in defining user functions for handling GIS. - I found this compliance test suite [7] for PostGIS that may come in handy. Also, I have a few other questions related to how should I proceed: - I have already read the introductory developers guide [8], the contributing guidelines [9] and will submit the MariaDB Contributor Agreement [10] as soon as possible. Is there anything else I should do before getting started? - I found no template for Google Summer of Code applications. Is there anything in particular that mentors would like to see? I was thinking of organizing my application in multiple sections, some of the most important being: Synopsis, Benefits to the Users, Project Details, Deliverables, Project Schedule and my resume. I believe that I will receive more feedback related after I will have submitted my application. - Most issues already have an assignee. Is it fine if I try and send pull requests for some of those bugs? I could not decide between whether I should send this mail to discuss or developers mailing list. Hopefully, I chose the right one. I am sorry for my long email and thank you for attention. I am looking for feedback, so please do not hesitate to send me any suggestion you might have. [1] https://mariadb.com/kb/en/mariadb/google-summer-of-code-2016/ [2] https://jira.mariadb.org/browse/CONC-125 [3] https://jira.mariadb.org/browse/MDEV-5813 [4] http://www.opengeospatial.org/standards/sfs [5] http://portal.opengeospatial.org/files/?artifact_id=25354 [6] http://www.opengeospatial.org/compliance/ [7] https://svn.osgeo.org/postgis/trunk/extras/ogc_test_suite/README [8] https://mariadb.org/get-involved/getting-started-for-developers/ [9] https://mariadb.com/kb/en/mariadb/contributing-code/ [10] https://mariadb.com/kb/en/mariadb/mca/ Best regards, Dan Ungureanu

2 1

[Maria-developers] GSoC 2016 Maxscale Luafilter
by Pedro Moratta 08 Mar '16

08 Mar '16

Hi, My name in Pedro Mora and I am a MS Computer Science student at Milano Institute of technology. I wish to work on Maxscale filters during GSoC 2016. I am interested in working on the Lua filter because I am familiar with Lua scripting and C/C++ programming as well as working Knowledge of Maxscale. I have been looking at some sample filters but could not find the luafilter mentioned as it seems the github branch https://github.com/mariadb-corporation/MaxScale/blob/luafilter/server/modul… <https://github.com/mariadb-corporation/MaxScale/blob/luafilter/server/modul…> no longer exists. Please I would love to know where the current repo for the Luafilter is located so as to see what had been done and what we would need to enhance this summer. I look forward to feedback soon. Regards, Pedro

1 0

[Maria-developers] Enhancing mysqlbinlog (MDEV-4989 and MDEV-5993)
by Dan Ungureanu 08 Mar '16

08 Mar '16

Hello, I have been looking into MDEV-4989 [1] and MDEV-5993 [2] and as far as I know, the features requested in those two tickets have already been implemented in MySQL. However, the team decided to approach this problem in a different manner and focus more on parallel replication so the GTID implementations diverged. Now, my question is: for starters, is it enough to consider the code from MySQL as some sort of guideline when implementing these features in MariaDB? Kristian Nielsen did a great job explaining GTID in his blog posts and even though the mechanism behind the scene seems to be pretty simple, the codebase is quite massive and I have a hard time going through all of it. >From what I can understand so far, I do not believe that porting the code from MySQL to MariaDB is possible since there are a few key aspects that differ. Is that correct? I hope I made myself clear and I look forward to hearing from you. [1] https://jira.mariadb.org/browse/MDEV-4989 [2] https://jira.mariadb.org/browse/MDEV-5993 Best regards, Dan Ungureanu

2 1

Re: [Maria-developers] [Commits] e19e45b: MDEV-9651 - Simplify audit event dispatching
by Sergei Golubchik 07 Mar '16

07 Mar '16

Hi, Sergey! On Mar 01, Sergey Vojtovich wrote: > revision-id: e19e45b23c07ef8d0679ca0ed7d93b70625e1a26 (mariadb-10.1.8-141-ge19e45b) > parent(s): e9f6c8167e238107643b9ef6228ca23976afb654 > committer: Sergey Vojtovich > timestamp: 2016-03-01 10:21:42 +0400 > message: > > MDEV-9651 - Simplify audit event dispatching > > Simplified audit event dispatching call chain from: Ok to push. Btw, you there's a conveniency helper safe_strlen() that you could've used. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] GSoC 2016 Maxscale Filter Implementation.
by Jason Kravitz 07 Mar '16

07 Mar '16

Hi, I am Jay Kravitz an Informations systems student at Georgetown University and I would love to work on Maxscale during GSoC 2016. I am interested in the Filter projects specifically the filter to capture incoming operations for consumption in external sources. I have build Maxscale on Ubuntu 15 and I am currently looking at some filters in the codebase to understand the filter API. I will also submit a backup proposal for the query injection filter hoping I am given the opportunity to implement one. I look forward to hearing from anyone soon. Regards, JK.

1 0

[Maria-developers] main.delayed fails on timeout
by Alexander Barkov 03 Mar '16

03 Mar '16

Hello Monty, Sergei, Can you please have a look into: http://buildbot.askmonty.org/buildbot/builders/kvm-deb-precise-amd64/builds… Can you please give any suggestions about what we can do to fix the problem with main.delayed failing sporadically? This problem is blocking us badly in Debian. Otto can try to disable this test temporary. But it would be nice to have a real fix. Any tips would be very appreciated. Thanks!

2 1

[Maria-developers] GSoC 2015 mysqlbinlog Enhancement.
by Rebecca Tucker 03 Mar '16

03 Mar '16

Hello everyone, I am Rebecca Tucker a MS student at University of Maryland and I wish to participate in the summer of code to work on making the mysqlbinlog tool to understand global transaction IDs. I am familiar with the tool and replication and can write documented C and C++ code. I Look forward to communicating with you soon to get more pointers on the project. Cheers, Becca.

1 0

[Maria-developers] GSOC 2016 self Introduction
by Tapan Halani 02 Mar '16

02 Mar '16

Hello all devs, I am Tapan Halani , an undergrad student from India. I am interested in contributing code to MariaDB and am taking part in GSoC 2016 also. I have already done all the prerequisite code cloning and setup etc. and am going through MDEV-4989 , trying to understand and solve it. Looking forward to have a great time contributing. Thanks Tapan

1 0

[Maria-developers] Igor please review: b6a2441: Remove all comments starting with 'psergey'
by Sergey Petrunia 02 Mar '16

02 Mar '16

Please review the below: ----- Forwarded message from Sergei Petrunia <psergey(a)askmonty.org> ----- Date: Wed, 2 Mar 2016 13:35:11 +0300 (FET) From: Sergei Petrunia <psergey(a)askmonty.org> To: commits(a)mariadb.org, sergey(a)mariadb.com Subject: b6a2441: Remove all comments starting with 'psergey' revision-id: b6a2441edf3bfce381f5dc24f5be23fce9ac36dd parent(s): b05158cc10a75196b5c0bf8dad9360608a2dd5b9 committer: Sergei Petrunia branch nick: 10.1-dbg5 timestamp: 2016-03-02 13:35:11 +0300 message: Remove all comments starting with 'psergey' --- sql/item.cc | 3 +-- sql/item_subselect.cc | 15 +-------------- sql/opt_range.cc | 4 ++-- sql/opt_subselect.cc | 1 - sql/opt_table_elimination.cc | 2 -- sql/sql_base.cc | 4 ++-- sql/sql_join_cache.cc | 7 +------ sql/sql_select.cc | 39 ++------------------------------------- sql/sql_statistics.cc | 9 --------- sql/sql_union.cc | 2 +- 10 files changed, 10 insertions(+), 76 deletions(-) diff --git a/sql/item.cc b/sql/item.cc index d6132cc..8fd45d5 100644 --- a/sql/item.cc +++ b/sql/item.cc @@ -4227,8 +4227,7 @@ static bool mark_as_dependent(THD *thd, SELECT_LEX *last, SELECT_LEX *current, DBUG_PRINT("info", ("mark_item: %p lex: %p", mark_item, last)); mark_item->depended_from= last; } - if (current->mark_as_dependent(thd, last, - /** resolved_item psergey-thu **/ mark_item)) + if (current->mark_as_dependent(thd, last, mark_item)) DBUG_RETURN(TRUE); if (thd->lex->describe & DESCRIBE_EXTENDED) { diff --git a/sql/item_subselect.cc b/sql/item_subselect.cc index a5ef81e..e11eafe 100644 --- a/sql/item_subselect.cc +++ b/sql/item_subselect.cc @@ -243,10 +243,6 @@ bool Item_subselect::fix_fields(THD *thd_param, Item **ref) done_first_fix_fields= TRUE; inside_first_fix_fields= TRUE; upper_refs.empty(); - /* - psergey-todo: remove _first_fix_fields calls, we need changes on every - execution - */ } eliminated= FALSE; @@ -1753,7 +1749,6 @@ my_decimal *Item_in_subselect::val_decimal(my_decimal *decimal_value) Check that the right part of the subselect contains no more than one column. E.g. in SELECT 1 IN (SELECT * ..) the right part is (SELECT * ...) */ - // psergey: duplicated_subselect_card_check if (select_lex->item_list.elements > 1) { my_error(ER_OPERAND_COLUMNS, MYF(0), 1); @@ -2172,7 +2167,6 @@ bool Item_allany_subselect::is_maxmin_applicable(JOIN *join) DBUG_ENTER("Item_in_subselect::row_value_transformer"); - // psergey: duplicated_subselect_card_check if (select_lex->item_list.elements != cols_num) { my_error(ER_OPERAND_COLUMNS, MYF(0), cols_num); @@ -3600,7 +3594,6 @@ void subselect_engine::set_row(List<Item> &item_list, Item_cache **row) if (!(row[i]= Item_cache::get_cache(thd, sel_item, sel_item->cmp_type()))) return; row[i]->setup(thd, sel_item); - //psergey-backport-timours: row[i]->store(sel_item); } if (item_list.elements > 1) cmp_type= res_type= ROW_RESULT; @@ -4017,9 +4010,6 @@ int subselect_uniquesubquery_engine::index_lookup() subselect_uniquesubquery_engine::~subselect_uniquesubquery_engine() { - /* Tell handler we don't need the index anymore */ - //psergey-merge-todo: the following was gone in 6.0: - //psergey-merge: don't need this after all: tab->table->file->ha_index_end(); } @@ -4172,9 +4162,6 @@ int subselect_indexsubquery_engine::exec() uint subselect_single_select_engine::cols() { - //psergey-sj-backport: the following assert was gone in 6.0: - //DBUG_ASSERT(select_lex->join != 0); // should be called after fix_fields() - //return select_lex->join->fields_list.elements; return select_lex->item_list.elements; } @@ -5136,7 +5123,7 @@ void check_out_index_stats(JOIN *join) table_map key_used=0; table_map non_key_used= 0; - bzero(&key_start_use, sizeof(key_start_use)); //psergey-todo: safe initialization! + bzero(&key_start_use, sizeof(key_start_use)); bzero(&key_infix_use, sizeof(key_infix_use)); for (order= join->group_list; order; order= order->next) diff --git a/sql/opt_range.cc b/sql/opt_range.cc index ed7e9a5..72a851d 100644 --- a/sql/opt_range.cc +++ b/sql/opt_range.cc @@ -8640,7 +8640,7 @@ static bool remove_nonrange_trees(RANGE_OPT_PARAM *param, SEL_TREE *tree) if (key1->elements != 1) { - key2->use_count+=key1->elements-1; //psergey: why we don't count that key1 has n-k-p? + key2->use_count+=key1->elements-1; key2->increment_use_count((int) key1->elements-1); } if (key1->type == SEL_ARG::MAYBE_KEY) @@ -10104,7 +10104,7 @@ ha_rows check_quick_select(PARAM *param, uint idx, bool index_only, } DBUG_PRINT("exit", ("Records: %lu", (ulong) rows)); - DBUG_RETURN(rows); //psergey-merge:todo: maintain first_null_comp. + DBUG_RETURN(rows); } diff --git a/sql/opt_subselect.cc b/sql/opt_subselect.cc index cb356fb..1a13dd6 100644 --- a/sql/opt_subselect.cc +++ b/sql/opt_subselect.cc @@ -635,7 +635,6 @@ int check_and_do_in_subquery_rewrites(JOIN *join) (oe1, oe2) IN (SELECT ie1, ie2, ie3 ...) TODO why do we have this duplicated in IN->EXISTS transformers? - psergey-todo: fix these: grep for duplicated_subselect_card_check */ if (select_lex->item_list.elements != in_subs->left_expr->cols()) { diff --git a/sql/opt_table_elimination.cc b/sql/opt_table_elimination.cc index 912ef4a..8378e8e 100644 --- a/sql/opt_table_elimination.cc +++ b/sql/opt_table_elimination.cc @@ -757,8 +757,6 @@ void eliminate_tables(JOIN *join) else { DBUG_ASSERT(!tbl->nested_join || tbl->sj_on_expr); - //psergey-todo: is the following really correct or we'll need to descend - //down all ON clauses: ? if (tbl->sj_on_expr) tables_used_on_left |= tbl->sj_on_expr->used_tables(); } diff --git a/sql/sql_base.cc b/sql/sql_base.cc index ac2162b..3d02f9c 100644 --- a/sql/sql_base.cc +++ b/sql/sql_base.cc @@ -3965,7 +3965,7 @@ thr_lock_type read_lock_type_for_table(THD *thd, goto end; } DBUG_PRINT("tcache", ("opening table: '%s'.'%s' item: %p", - tables->db, tables->table_name, tables)); //psergey: invalid read of size 1 here + tables->db, tables->table_name, tables)); (*counter)++; /* @@ -8105,7 +8105,7 @@ bool setup_tables(THD *thd, Name_resolution_context *context, Item *item= table_list->jtbm_subselect->optimizer; if (table_list->jtbm_subselect->optimizer->fix_fields(thd, &item)) { - my_error(ER_TOO_MANY_TABLES,MYF(0), static_cast<int>(MAX_TABLES)); /* psergey-todo: WHY ER_TOO_MANY_TABLES ???*/ + my_error(ER_TOO_MANY_TABLES,MYF(0), static_cast<int>(MAX_TABLES)); DBUG_RETURN(1); } DBUG_ASSERT(item == table_list->jtbm_subselect->optimizer); diff --git a/sql/sql_join_cache.cc b/sql/sql_join_cache.cc index f84440a..9f69169 100644 --- a/sql/sql_join_cache.cc +++ b/sql/sql_join_cache.cc @@ -3919,11 +3919,6 @@ int JOIN_TAB_SCAN_MRR::next() If a record in in an incremental cache contains no fields then the association for the last record in cache will be equal to cache->end_pos */ - /* - psergey: this makes no sense where HA_MRR_NO_ASSOC is used. - DBUG_ASSERT(cache->buff <= (uchar *) (*ptr) && - (uchar *) (*ptr) <= cache->end_pos); - */ if (join_tab->table->vfield) update_virtual_fields(join->thd, join_tab->table); } @@ -4591,7 +4586,7 @@ bool JOIN_CACHE_BKAH::prepare_look_for_matches(bool skip_last) { last_matching_rec_ref_ptr= next_matching_rec_ref_ptr= 0; if (no_association && - !(curr_matching_chain= get_matching_chain_by_join_key())) //psergey: added '!' + !(curr_matching_chain= get_matching_chain_by_join_key())) return 1; last_matching_rec_ref_ptr= get_next_rec_ref(curr_matching_chain); return 0; diff --git a/sql/sql_select.cc b/sql/sql_select.cc index 4759af0..878cb8b 100644 --- a/sql/sql_select.cc +++ b/sql/sql_select.cc @@ -2861,9 +2861,6 @@ void JOIN::exec_inner() /* Free first data from old join */ - /* - psergey-todo: this is the place of pre-mature JOIN::free call. - */ curr_join->join_free(); if (curr_join->make_simple_join(this, curr_tmp_table)) DBUG_VOID_RETURN; @@ -3001,7 +2998,6 @@ void JOIN::exec_inner() curr_join->select_distinct=0; } curr_tmp_table->reginfo.lock_type= TL_UNLOCK; - // psergey-todo: here is one place where we switch to if (curr_join->make_simple_join(this, curr_tmp_table)) DBUG_VOID_RETURN; calc_group_buffer(curr_join, curr_join->group_list); @@ -6646,12 +6642,6 @@ static void choose_initial_table_order(JOIN *join) jtab_sort_func= straight_join ? join_tab_cmp_straight : join_tab_cmp; } - /* - psergey-todo: if we're not optimizing an SJM nest, - - sort that outer tables are first, and each sjm nest follows - - then, put each [sjm_table1, ... sjm_tableN] sub-array right where - WHERE clause pushdown would have put it. - */ my_qsort2(join->best_ref + join->const_tables, join->table_count - join->const_tables, sizeof(JOIN_TAB*), jtab_sort_func, (void*)join->emb_sjm_nest); @@ -9171,7 +9161,6 @@ static bool create_ref_for_key(JOIN *join, JOIN_TAB *j, !(parent->join_tab_reexec= (JOIN_TAB*) thd->alloc(sizeof(JOIN_TAB)))) DBUG_RETURN(TRUE); /* purecov: inspected */ - // psergey-todo: here, save the pointer for original join_tabs. join_tab= parent->join_tab_reexec; table= &parent->table_reexec[0]; parent->table_reexec[0]= temp_table; table_count= top_join_tab_count= 1; @@ -10060,9 +10049,6 @@ bool TABLE_LIST::is_active_sjm() DBUG_ASSERT(inner_tab->table); current_map= inner_tab->table->map; used_tables2|= current_map; - /* - psergey: have put the -1 below. It's bad, will need to fix it. - */ COND *tmp_cond= make_cond_for_table(thd, on_expr, used_tables2, current_map, /*(inner_tab - first_tab)*/ -1, @@ -11070,11 +11056,6 @@ void check_join_cache_usage_for_tables(JOIN *join, ulonglong options, idx, prev_tab); tab->use_join_cache= MY_TEST(tab->used_join_cache_level); - /* - psergey-merge: todo: raise the question that this is really stupid that - we can first allocate a join buffer, then decide not to use it and free - it. - */ if (join->return_tab) { tab= join->return_tab; @@ -15622,12 +15603,6 @@ bool cond_is_datetime_is_null(Item *cond) @retval false cannot be used */ -/* - psergey-todo: this returns false for int_column='1234' (here '1234' is a - constant. Need to discuss this with Bar). - - See also Field::test_if_equality_guaranees_uniqueness(const Item *item); -*/ static bool test_if_equality_guarantees_uniqueness(Item *l, Item *r) { @@ -18293,10 +18268,6 @@ enum_nested_loop_state rc= evaluate_join_record(join, join_tab, error); } - /* - Note: psergey has added the 2nd part of the following condition; the - change should probably be made in 5.1, too. - */ bool skip_over= FALSE; while (rc == NESTED_LOOP_OK && join->return_tab >= join_tab) { @@ -20956,7 +20927,7 @@ uint find_shortest_key(TABLE *table, const key_map *usable_keys) !(table->file->index_flags(best_key, 0, 1) & HA_CLUSTERED_INDEX))) goto use_filesort; - if (select && // psergey: why doesn't this use a quick? + if (select && table->quick_keys.is_set(best_key) && best_key != ref_key) { key_map tmp_map; @@ -24002,11 +23973,7 @@ void JOIN_TAB::save_explain_data(Explain_table_access *eta, eta->type= tab_type; /* Build "possible_keys" value */ - // psergey-todo: why does this use thd MEM_ROOT??? Doesn't this - // break ANALYZE ? thd->mem_root will be freed, and after that we will - // attempt to print the query plan? append_possible_keys(thd->mem_root, eta->possible_keys, table, keys); - // psergey-todo: ^ check for error return code /* Build "key", "key_len", and "ref" */ if (tab_type == JT_NEXT) @@ -24054,8 +24021,6 @@ void JOIN_TAB::save_explain_data(Explain_table_access *eta, eta->hash_next_key.set(thd->mem_root, & table->key_info[index], table->key_info[index].key_length); - // psergey-todo: ^ is the above correct? are we necessarily joining on all - // columns? } if (!key_info) @@ -24211,7 +24176,7 @@ void JOIN_TAB::save_explain_data(Explain_table_access *eta, eta->push_extra(ET_OPEN_FRM_ONLY); else eta->push_extra(ET_OPEN_FULL_TABLE); - /* psergey-note: the following has a bug.*/ + if (table_list->is_table_read_plan->trivial_show_command || (table_list->is_table_read_plan->has_db_lookup_value() && table_list->is_table_read_plan->has_table_lookup_value())) diff --git a/sql/sql_statistics.cc b/sql/sql_statistics.cc index 28e76ae..b22a2ae 100644 --- a/sql/sql_statistics.cc +++ b/sql/sql_statistics.cc @@ -3650,15 +3650,6 @@ double get_column_range_cardinality(Field *field, { double avg_frequency= col_stats->get_avg_frequency(); res= avg_frequency; - /* - psergey-todo: what does check for min_value, max_value mean? - min/max_value are set to NULL in alloc_statistics_for_table() and - alloc_statistics_for_table_share(). Both functions will immediately - call create_min_max_statistical_fields_for_table and - create_min_max_statistical_fields_for_table_share() respectively, - which will set min/max_value to be valid pointers, unless OOM - occurs. - */ if (avg_frequency > 1.0 + 0.000001 && col_stats->min_value && col_stats->max_value) { diff --git a/sql/sql_union.cc b/sql/sql_union.cc index c835083..b369ef0 100644 --- a/sql/sql_union.cc +++ b/sql/sql_union.cc @@ -974,7 +974,7 @@ bool st_select_lex_unit::exec() 1st execution sets certain members (e.g. select_result) to perform subquery execution rather than EXPLAIN line production. In order to reset them back, we re-do all of the actions (yes it is ugly): - */ // psergey-todo: is the above really necessary anymore?? + */ join->init(thd, item_list, fake_select_lex->options, result); saved_error= mysql_select(thd, &fake_select_lex->ref_pointer_array, &result_table_list, ----- End forwarded message ----- -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] GSoC 2016 self Introduction
by Charles Muurmu 02 Mar '16

02 Mar '16

Dear MariaDB Developers, I am Charles Muurmu and I wish to participate in GSoC 2016 (and 17 if possible) with MariaDB. First, congratulations for passing a successful application with Google. I have gone through the program time line and students applications start in two weeks, However I am interested in two projects for which I will submit 2 proposals. First, I am interested in the Cassandra storage engine V2 as I have basic knowledge on how to write storage engine plugins as well as working knowledge of cassandra. Also I will be having as backup project the GIS enhancement task which requires adding a new geometry type containing the third coordinate, writing importers and exporters for the extended geometry formats... I have two books guiding me for now: Understanding MySQL Internals by O'reilly media and MySQL plugin Development by Sergei Golubchik. While preparing for student application I will be reading mariadb Knowledge base and ask questions on #maria. I will also try to look an the DataStax C++ Driver which will be used. Thanks, Charles.

1 0

[Maria-developers] [GSoC2016] Contribution to Cassandra SE v2
by Olzhas Kaiyrakhmet 02 Mar '16

02 Mar '16

Hello, I am Olzhas Kaiyrakhmet, computer science student at Ulsan National Institute of Science and Technology, Korea. I am interested in contributing to code to MariaDB and participating in GSoC2016. I do find the project, Cassandra SE v2, very interesting for me. Recently, I worked with Cassandra cluster and did look inside of it code. Tried to do some modifications. Also, I have some experience on contributing to distributive systems and databases as research intern in university. I have a great will to contribute for this project with your assistance. Best wishes, Olzhas kaiyrakhmet

3 2

[Maria-developers] Time delayed replication
by Colin Charles 29 Feb '16

29 Feb '16

Hi! Quick question — we’ve had https://jira.mariadb.org/browse/MDEV-8341 open for quite sometime now, and MySQL 5.6 and 5.7 obviously implement this feature, time delayed replication Are you using it in production? Are you finding that you are missing it in MariaDB Server? How important would it be to you, to have this in MariaDB Server 10.2? Thanks -- Colin Charles, http://bytebot.net/blog/ twitter: @bytebot | skype: colincharles "First they ignore you, then they laugh at you, then they fight you, then you win." -- Mohandas Gandhi

2 1

Re: [Maria-developers] A proposal to deprecate syntax: SELECT a'test'
by Michael Widenius 29 Feb '16

29 Feb '16

Hi! > Hello Monty, all, > > > We discussed this with Sergei and both think that it will be a good idea > to deprecate this syntax: > > SELECT a'test'; > > where 'a' is an identifier and 'test' its alias. Is this dependent on if there is a space or not between a and 'test' ? Note that in MySQL and MariaDB, ' and " are both used for quoting strings and I don't think we can change that without affecting a lot of applications! I think it's ok to require one to have a space between a and 'test', if that would help at all. > This is a non-standard way, and it conflicts with some other important > standard SQL grammar. See below. Note that in MySQL/MariaDB, when not using ANSI mode, ' and " are equal and ` is used for identifiers, so this is expected to be non-standard. For quoting identifiers ` should be used. Of course, if you are using ANSI mode, then things should be as close to ANSI as possible. > Typified literals and a syntax conflict > --------------------------------------- > > An identifier followed by a single-quoted text string is needed > for typified literals: > > SELECT INET6'ffff::ffff'; > > We already support this for temporal literals, which is a part of > the SQL standard: > > SELECT TIME'10:10:10'; > SELECT TIMESTAMP'2001-01-01 10:20:30'; > SELECT DATE'2001-01-01'; > > Note, the conflict already exists in MySQL and MariaDB. > > > This script: > > DROP TABLE IF EXISTS t1; > CREATE TABLE t1 (date date); > INSERT INTO t1 VALUES ('2016-02-19'); > SELECT date'2001-01-01' FROM t1; > > returns: > > +------------------+ > | date'2001-01-01' | > +------------------+ > | 2001-01-01 | > +------------------+ > > Notice, it returns a DATE literal, it does not return the value of > the column t1.date using '2001-01-01' as an alias. As ansi sql doesn't allow you to use date as a column name, this isn't really as big problem as it may seam ;) Anyway, I agree that this can be confusing, but so is having date as a column namn. > Notice the difference: > > SELECT date`2001-01-01` FROM t1; > +------------+ > | 2001-01-01 | > +------------+ > | 2016-02-19 | > +------------+ > > This is confusing. Not really, as you are using different quotes and you get the expected answer. This should be same as if you used " in ANSI SQL, isn't it? > Shift-reduce conflicts > ---------------------- > By the way, the fact that we support single-quoted string as an alias > is the reason for shift-reduce conflicts in sql_yacc.yy, because > TIME/DATE/TIMESTAMP followed by 'string' can be interpreted in two ways: > - a column name followed by an alias > - an SQL-standard time literal What is the SQL-standard way to use an alias? (Sorry, can't access the standard just now): SELECT date "2001-01-01" FROM t1; or SELECT date '2001-01-01' FROM t1; And if you execute both of these queries, will you not get the same confusing result for one of them? As ' and " are identical in MySQL mode, I don't see how you can fix this without causing even more confusion. > What can be used instead > ------------------------ > We have a number of other ways to specify aliases: > > The standard ways: > > SELECT a AS test; > SELECT a "test"; > SELECT a AS "test"; Ok, this answers my question above. The standard says that an alias is a text string. In MySQL / MariaDB a text string can be specified with both ' or ". I don't think it's good idea to make alias a special case where you have to specify the text string only with ". > MySQL/MariaDB extensions: > > > SELECT a `test`; > SELECT a AS `test`; > > That should be enough. Note that this has nothing exclusively to do with alias, but with the MySQL/MariaDB extension that you can use ' instead of " as a string delimiter. > Proposal > -------- > 1. In 10.1 we add a warning when a single-quoted string is used as a > column alias. Something like this should work: > > Single quotes in a select sublist alias are deprecated. Use double > quotes instead. This would break the specification of what a text strings is in MariaDB. It would also be very confusing that text strings needs to be specified differently in different places of the SQL syntax. > 2. In 10.2 we disallow this syntax by default and add either sql_mode or > old_mode to enable the old behavior. As you can already get the behavior you want by specifying ANSI MODE, I don't think this change is necessary. I think it's also bad as it makes it harder to define what a text string is. For example: In MariaDB, a text string can be delimited with ' or ", except in the case of alias when you have to use ' I think it's better to require, as most ANSI SQL databases does, one to quote column names like 'date' and 'time' to ensure they are not mixed with operators or string prefixes. Regards, Monty

2 3

[Maria-developers] Type_handler and INET6
by Alexander Barkov 29 Feb '16

29 Feb '16

Hello Sergei, I'm sending a new version for Type_handler and INET6 for your review. There are three patches attached. 1. inet6-v65-prepare.diff.gz Includes preparatory changes, to avoid different execution paths for different data types. 2. inet6-v65-incremental.diff.gz Actually adds INET6, together with val_raw family methods, last minute changes that did not get into #1. 3. inet6-v65-full.diff.gz is a full patch that includes the previous two. Details on the patches as well as my answers to your previous review suggestions follow below. Thanks. The first patch makes preparatory changes, to avoid different execution paths and adds some helper classes needed for the API. - Adds a class Record_addr, to simplify API (pass just one value instead of 3 separate values for ptr, null_ptr and null_bit) and share some code moving initialization (e.g. for a temporary table purposes) to Record_addr constructors. Derives Field from it. - Adds a class Type_ext_attributes. This collects extra members of Item_type_holder. Currently it covers only extra attributes for GEOMETRY, DECIMAL, ENUM/SET. We'll add a generic "void *extra" pointer here, when user-defined types need even more extra attributes. This class will also be needed inside Item_hybrid_func, which currently have a bug (do not preserve geometry type and srid of the arguments). I'll fix it later. See MDEV-9405. - Adds a class Create_attr, to pass Create_field related attributes to handlers. - Adds a class Type_handler_register, to look up handlers by type code and names. - Continues unification of the code that creates temporary fields for various Items, see ::create_tmp_field() in sql_select.cc. This task is not fully finished. Some more chages are still needed. The intent is to get rid of ::create_tmp_field() and switch to use methods in Item instead. - Adds a data type handler pointer to Arg_comparator. Previously the was only "Item_result m_compare_type". - Splits Item_func_case::fix_length_and_dec() into three separate steps: fix_field_type(), fix_return_arguments(), fix_comparison_arguments(), to incorporate the type handler methods easier. - Splits Item_func_between::val_int() into virtual methods in Type_handler_xxx. - Adds virtual methods to Type_handler, to avoid different execution paths for non-traditional types: calc_pack_length(), check_column_definition(), prepare_column_definition(), make_table_field(), Item_save_in_field(), make_cache_item(), calc_pack_length(), calc_display_length(), Item_type_holder_join_attributes(), Item_func_hex_val_str_ascii(), Item_func_hybrid_field_type_val_xxx(), Item_func_between_fix_length_and_dec(), Item_func_between_val_int(), Item_sum_hybrid_fix_length_and_dec(), - Adds Type_handler_numeric, to share some code in real, int decimal handlers. - Moves field_type_for_temporal_comparison() from Item to Type_handler. - Makes some other changes in data type aggregation in various pieces of the code. This will need some more changes though. - Removes Field::result_merge_type(). Uses Type_handler::cmp_type() and Type_handler::result_type() instead. - Simplifies ::make_field() in field.cc, using Type_handler::make_table_field(). - Removes the global Arg_comparator::comparator_matrix[6][2] and uses methods in Type_handler instead. The second patch actually adds INET6 and: - Adds val_raw family methods into Items - Adds some pieces into the type aggregation code in a few places - Fixes sql_yacc.yy to handle non-traditional types, adds Cast_type and fixes Lex_cast_type_st. - Fixes Type_handler_register to search non-traditional types by name. - Disables equal expression propagation for non-traditional types (the code is not ready yet for this, e.g. Item_equal needs some changes) - Adds the INET6 handler with tests Also, I answer your questions and suggestions that you made during the first review. Please see below. >> > > === added file 'mysql-test/r/type_inet6.result' > > > --- mysql-test/r/type_inet6.result 1970-01-01 00:00:00 +0000 > > > +++ mysql-test/r/type_inet6.result 2014-07-01 11:10:33 +0000 > > > +::ff > > > +::ffff > > > +::0.255.255.255 > > > +::255.255.255.255 > > > +::ff:ffff:ffff > > > +::ffff:255.255.255.255 > > Hmm. A bit unexpected. Do you have a link to some > document that specifies how ipv6 numbers should be parsed and printed? https://en.wikipedia.org/wiki/IPv6_address says: :ffff:0:0/96 — This prefix is designated as an IPv4-mapped IPv6 address. Note, I also checked this in PostgreSQL and it works in the same way: bar=> select inet'::ffff:ffff:ffff'; inet ------------------------ ::ffff:255.255.255.255 > > > +SELECT * FROM t1 WHERE a=0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF; > > > +a > > > +ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff > > > +ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff > > > +Warnings: > > > +Warning 1105 Cast to INET6 converted '\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF' to 'ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff' > > Too many warnings. I'd rather use Item_cache or something similar > to have only one actual type conversion and one warning It now creates Item_cache. So there's only one warning. > > > === modified file 'sql/field.h' > > > --- sql/field.h 2014-06-11 08:08:08 +0000 > > > +++ sql/field.h 2014-07-01 10:38:12 +0000 > > > @@ -366,6 +437,10 @@ class Field > > > virtual ~Field() {} > > > /* Store functions returns 1 on overflow and -1 on fatal error */ > > > virtual int store(const char *to, uint length,CHARSET_INFO *cs)=0; > > > + virtual int store_raw(const String *raw, enum_field_types type) > > > + { > > > + return store(raw->ptr(), raw->length(), raw->charset()); > > > + } > > How do you guarantee that the 'raw' is in the correct internal raw format? > Or is it only used after int_to_raw, str_to_raw, etc? raw representation can only be created by a method of the corresponding Type_handler_xxx, or come from "ptr" of the corresponding Field_xxx. Bad raw format means that there is a bug inside Type_handler_xxx. Btw, In the latest patch I changed it to: int store_raw_native(const char *str, uint length); Conversion (when needed) is done outside of Field. > > > +class Field_type_joiner > > It's rather "aggregator", not "joiner". > But in fact, it's not even that, it's a pair of { type, derivation }. > May be it should be Field_type_with_derivation ? :) > And a method Field_type_with_derivation::aggregate(...) It's now Type_handler_hybrid_field_type, which I earlier added in 10.1. There is no "derivation" in this version, as we discussed in Luxembough. Sigh, I'd really really prefer if we had it though. I think when field=expr is compared according to the field data type is clearer than they're compared accoring to the expr datatype. Also, comparing according to field will use the optimizer by default, without having to cast "expr" to the field data type. > > > +class Type_handler > > > +{ > > > +public: > > > + Type_handler() {} > > > + ~Type_handler() {} > > > + virtual const char *field_type_name() const = 0; > > > + virtual uint field_type_name_length() const = 0; > > may be, remove field_ prefix, or even a field_type_ prefix? > the method is still unambiguous, but much shorter and readable. > and, may be, make it non-virtual, like > > ... > private: > LEX_STRING type_name; > public: > Type_handler(const char *name_arg) > { > type_name.str= name_arg; > type_name.length= strlen(name_arg); > } > const char *name() const { return type_name.str; } > const char *name_length() const { return type_name.length; } > ... > > and your new type will do in the constructor: > > Type_handler_inet6(): Type_handler("INET6") { } > > less boilerplate code for the new types, and you know how I'm trying > to avoid boilerplate code in the APIs :) I'm using the very needed new class Name for it. It now looks simpler, I believe. > > > > + virtual enum_field_types field_type() const = 0; > > That doesn't work for loadable types. > I think that we eventually need to do with types what we've done with > storage engines. There was db_type enum, but now the engine is almost always > identified by the pointer to the handlerton. So, for types a pointer to > the Type_handler might be a unique type identifier in the server. > Replacing enum_field_types where possible. Sure, in the end we should get a code that will identify handlers by a pointer rather than field_type(). This will need a few additional steps, which I just did not have a chance to do yet: - Fix all existing Items to return type_handler() instead of field_type() as a type identifier. This will be easier after we add virtual inheritance (or templates) soon, on top of my Type_handler changes. - Get rid of all field_type() calls: split the remaining pieces of the code that use field_type() into more virtual methods in Type_handler. - Fix FRM to store new field types as a textual name rather than a numeric ID. Can we do these steps a separately? Note, it can be done after adding INET6 and UUID, before adding dynamic loadability. > > > + virtual enum_field_types fallback_field_type() const = 0; > > I don't like the idea of fallback_field_type. > You have Field_type_joiner for new types, but you fallback to > Field::field_type_merge() for old types. I'd rather not distinguish > between "old" and "new" types. Just like we have with storage engines or > authentication - there are no "native" and "plugin" storage engines (or > authentication), *all* storage engines are plugins. So there's no code like > > if (is_plugin(engine_type)) > ... > else > ... > > so, I prefer uniform code with no special checks for type handlers. > Implementations differ, but the interface is the same for all types. I nodified the code, so there is no a separate fallback_field_type() any more. But there is still cmp_type(), which is uses in very rare cases, to exchange data to/from other data types: INET6->something, something->INET6. Also, there are still some different execution paths inside the methods that aggregate data types. We'll need to discuss it. I could not come up with a simple single method for aggregation. Also, currently field type aggregation is done in different ways in various pieces of the code (hybrid functions, UNION, comparison). We should try to unify these ways. > > > + virtual Item_result cast_to_int_type() const = 0; > > Why is this abstract? It could have a reasonable default implementation > reducing the amount of boilerplate code in the Type_handler's > > > > + virtual int cmp(Item *a, Item *b) const = 0; > > > + virtual int cmp(String *a, String *b) const = 0; > > These too Not sure that exactly cast_to_int_type() can have a reasonable default. For INET6 and UUID it's DECIMAL_RESULT. For compressed URLs (MDEV-4630) and external ENUM (MDEV-4913) it's more likely to be STRING_RESULT. Anyway, I agree that we should add some reasonable defaults when possible. Perhaps we could add two abstract classes with some defaults: Type_handler | +-- Type_handler_traditional (derive all existing types from this) | +-- Type_handler_binary_fixed_length_storage (derive INET6 and UUID from this) But I'd propose not to hurry with that. We have time before we add loadabale data type plugins. Can we add UUID first? It will show how much INET6 and UUID can share and thus how much can be moved to Type_handler_binary_fixed_length_storage. > > > + virtual int save_in_field(Item_hex_hybrid *item, Field *field) const = 0; > > Why do you need that virtual? It could be always val_raw/store_raw I guess this question is not relevant any more, because we now have handlers for the traditional data types, and they need to treat hex hybrids differenty (as string for Field_str and Field_temporal, and as a longlong for the other datatypes). Note, I modified this piece slighly. It's now a virtual method in Field rather than in Type_handler: int Field::store_hex_hybrid(const char *str, uint length); > > > + virtual int set_cmp_func(Arg_comparator *cmp, Item *a, Item *b) const = 0; > > this can be removed, see below (under Type_handler_inet6::set_cmp_func) I could not get rid of it. Moreover, the traditional types now use this method. For example: bool Type_handler_int_result::set_comparator_func(Arg_comparator *cmp) const { return cmp->set_cmp_func_int(); } So now I have the same execution path for all data types here. > > > + virtual bool varbinary_to_raw(const String *from, String *to) const = 0; > > 1. This most certainly needs a comment. It took me quite a while to > understand what this method is for and why str_to_raw isn't used instead. > A comment should say that it's used when a field binary representation is > specified in a query directly was 0xHHHHHHHH... > > 2. Still, I'm not sure this method is needed. It means a different > behavior for > > where ipv6 = 0xHHHHHH... > > and > > create t1 (b binary(16)); insert t1 values (0xHHHHH....); > ... where ipv6 = a ... > Note, it's the same with what we do for numbers: CREATE OR REPLACE TABLE t1 (a INT, b BLOB); INSERT INTO t1 VALUES (0x31,0x31); SELECT COUNT(*) FROM t1 WHERE a=0x31; -- returns a row SELECT * FROM t1 WHERE a=b; -- returns no rows See below. > an alternative would be to treat any binary string as raw, > directly in str_to_raw. But a drawback would be that > > where ipv6 = "ffff::1" > > and > > where ipv6 = convert("ffff::1" to binary) > > will work differently. And it's not particularly good either. I like the idea to treat all binary strings as raw with INET6, and then with UUID. I think this is not really a drawback. Remember, when we discussed "is [VAR]BINARY a data type or a character set" around the year of 2005, we fortunately decided that they are data types, so we introduced data types [VAR]BINARY as a replacement to "CHAR CHARACTER SET binary". So [VAR]CHAR abd [VAR]BINARY are different data types! It should be OK that INET6 will communicate with two ***different*** datatypes [VAR]CHAR vs [VAR]BINARY differently. If we agree on that, I'll change the code accordingly. We preliminary agreed on IRC to make explicit and implicit conversion between a data type and BINARY use methods in Type_handler. (but I have not added them yet). > > So, it's a tradeoff, and the question is - what is "less gotcha" ? > I don't know. I have a slight preference towards removing varbinary_to_raw() > just to make the API smaller. And thinking about adding it if there would be > user complains. > > Another thought. I'm starting to doubt that we should support direct > raw values in the SQL at all. It might work for ipv6 > > where ipv6 = 0xHHHHHH > > but it certainly won't work for, say, integers: > > where int = 0xHHHHH > > ^^^ this is not raw, but a normal hexadecimal number. > So, unless you introduce a special unambiguous syntax (like RAW(0xHHHHH)), > I think that raw values should not be supported in SQL. This syntax looks useful for me: WHERE inet6_field=0xFFFF; Btw, in case of integer numbers, it's exactly the low level raw format in Field. This is how integer numbers are stored, isn't it? I didn't understand why it won't work for numbers. Note, comparison to a hex hybrid works fine for INET6 in my patch and does not break the existing data types. Numeric comparion goes through Item_hex_hybrid::val_int(). String comparison goes through Item_hex_hybrid::val_str(). INET6 comparison goes through Item_hex_hybrid::val_raw(). This is Arg_comparator::func who decides which method to use. In the future, we could probably unify this: implement val_raw() for the traditional data types, and use val_raw() for comparison instead of the usual methods like val_int(). It's more tricky what should happen in case of VARCHAR-alike or BLOB-alike data types (suppose we'll decide the standard CLOB type in the future). Should val_raw() include the length bytes? I think no, as it will assume duplication of "length", once in String::length(), and one more time in the beginning of String::ptr(). So I was thinking of "raw" as something very close to how the data is stored in Field, but it's not necessary a 100% match. Perhaps we just need a different name than "raw"... I agree we don't need the field-style low level raw format (e.g. with length bytes for a VARCHAR-alike data type). > > > + virtual Item *make_cast(MEM_ROOT *mem_root, Item *arg) const = 0; > > Hmm. This implies an Item_typecast_xxx for every type. > I would be easier to have one Item_typecast_handled, that stores > a pointer to Type_handler. Would be much easier than implementing > new Item_typecast_xxx for every type. Some data types could probably share the same Item_typecast_xxx, but I don't think it's generally doable with a single Item for all data types. There are a few reasons why I prefer a separate method in handler to create a CAST Item. - Different data types can have different set of extra attributes, which the corresponding Item_typecast_xxx should store in. It's easier to store attributes in typed structures rather than in a generic memory pointed by "void *". It will make our life easier when we start debugging in gdb :) - I plan to simplify sql_yacc.yy so the loadable datatypes and the traditional datatypes share the same CAST and field_type grammar. It will be needed soon even for the traditional data types. - A method in handler will help to add the missing CAST types easier, e.g. FLOAT, SMALLINT, etc. It looks natural that every Field_xxx has a corresponding Item_typecast_xxx. > > > + int store_raw(const String *str, enum_field_types type_arg) > > > + { > > > + if (type_arg != type()) > > > + { > > > + set_null(); > > > + return 1; // TODO: go through str/int/real/dec/date? > > > + } > > > + DBUG_ASSERT(str->length() == binary_length()); > > > + memcpy(ptr, str->ptr(), binary_length()); > > > + return 0; > > > + } > > val_raw and store_raw should not be virtual, they are always > identical for all types. Just don't forget the validity check. <offtopic> Btw, it now looks like this: int store_raw_native(const char *str, uint length) { DBUG_ASSERT(length == binary_length()); memcpy(ptr, str, binary_length()); return 0; } I moved type conversion to the upper level. </offtopic> Right, the actual "store" is often a simple memcpy() for fixed length data types like INET6. But it's not generally the case. - It can be different for non-fixed length data types. It's again about the fact that the value of val_raw() and the byte array stored in Field::ptr() can have slighly differrent formats. Variable length types will pass the values using a normal String format, without additional length bytes coded inside the array pointed by String::ptr(). - Also, in case we have a shared method to store, we'll have to have additional virtual methods for validity check, to call inside. There's no sense for two virtual calls. - Also, in some cases it will be possible to use val_raw() instead of val_int(), val_decimal(), val_str(), val_real(), get_date(), and thus share almost duplicate code that's now implemented for different Item_result types. I already mentioned Arg_comparator. But there could also be only one universal Item_cache, etc. > > > + int store_time_dec(MYSQL_TIME *ltime, uint dec) > > > + { > > > + my_decimal tmp; > > > + return store_decimal(date2my_decimal(ltime, &tmp)); > > > + } > > Numerous val_xxx and store_xxx could be replaced by xxx_to_raw > and raw_to_xxx + val_raw and store_raw. You shouldn't introduce > both val_xxx and raw_to_xxx. Same for store_xxx and xxx_to_raw Right. I don't expose raw_to_xxx publically any more. They are now internal protected Type_handler_inet6 routines. > > > > + > > > + uint size_of() const { return sizeof(*this); } > > > + }; > > > + > > > + > > > + class Arg_comparator_inet6: public Arg_comparator > > > + { > > > + public: > > > + int compare_inet6() > > > + { > > > + if ((*a)->val_raw(&value1, cmp_field_type()) || > > > + (*b)->val_raw(&value2, cmp_field_type())) > > > + goto null; > > > + if (set_null) > > > + owner->null_value= 0; > > > + DBUG_ASSERT(value1.length() == Type_handler_inet6::binary_length()); > > > + DBUG_ASSERT(value2.length() == Type_handler_inet6::binary_length()); > > > + return memcmp(value1.ptr(), value2.ptr(), Type_handler_inet6::binary_length()); > > > + null: > > > + if (set_null) > > > + owner->null_value= 1; > > > + return -1; > > > + } > > > + int compare_e_inet6() > > > + { > > > + bool a_is_null= (*a)->val_raw(&value1, cmp_field_type()); > > > + bool b_is_null= (*b)->val_raw(&value2, cmp_field_type()); > > > + if (a_is_null || b_is_null) > > > + return MY_TEST(a_is_null == b_is_null); > > > + DBUG_ASSERT(value1.length() == Type_handler_inet6::binary_length()); > > > + DBUG_ASSERT(value2.length() == Type_handler_inet6::binary_length()); > > > + return MY_TEST(memcmp(value1.ptr(), value2.ptr(), Type_handler_inet6::binary_length()) == 0); > > > + } > > we should try to get rid of one of the functions, see below > ... > ... > ... > > > + int set_cmp_func(Arg_comparator *cmp, Item *a, Item *b) const > > > + { > > > + cmp->set_func(cmp->is_owner_equal_func() ? > > > + (arg_cmp_func) &Arg_comparator_inet6::compare_e_inet6 : > > > + (arg_cmp_func) &Arg_comparator_inet6::compare_inet6, > > > + field_type()); > > Again, boilerplate code, better: > > cmp->set_func(Arg_comparator_inet6::compare_e_inet6, > Arg_comparator_inet6::compare_inet6, > field_type()); > > no "cmp->is_owner_equal_func() ?" - this can be done inside set_func() > Also, I wonder, whether every type really needs two functions, for equal func > and not. Perhaps having only one - compare_e_inet6() should be sufficient and > checks for NULL can be done in the common wrapper? > And, even more, set_cmp_func() is not necessary at all. Type_handler > should only have compare_type() and compare_e_type() virtual methods > (or only one compare_e_type()) and the implementation of set_cmp_func() > becomes identical for all types, and thus doesn't have to be virtual > anymore. One method less to implement. I agree. We should try to do that. Let's try to decompose the functions of the existing data types and the INET6 related functions in a separate patch. > > > > + return 0; > > > + } > > > + void sortlength(struct st_sort_field *order, const Item *item) const > > > + { > > > + order->result_type= item->cmp_type(); > > > + order->length= Type_handler_inet6::binary_length(); > > > + order->suffix_length= 0; > > > + order->need_strxnfrm= 0; > > > + } > > > + bool make_sort_key(struct st_sort_field *order, uchar *to) const > > > + { > > > + Item *item= order->item; > > > + String buf((char *) to, order->length, &my_charset_bin); > > > + /* > > > + make_sortkey() uses str_result(), val_result(), val_int_result(), > > > + val_decimal_result() for the standard result types. > > > + QQ: should we introduce val_raw_result() and use here instead of > > > + val_raw() ? > > Or, perhaps, we should fix make_sortkey() to use val_str, val_real, etc. > I failed to understant why *result() methos are used there. So did I. But as I had to implement the val_raw_result() methods anyway for other purposes, it was easy to use val_raw_result() here, for symmetry with the other types, and in the latest patch I still use _result_ here. It can be fixed later for all data types to use the usual val_xxx() instead. > I've even replaced them with non-*result versions and not a single test > failed. Including the test case that was added when val* methods > were replaced with val*result methods. > > > > + > > > +Type_handler_register::Type_handler_register() > > > +{ > > > + memset(m_handler, 0, sizeof(m_handler)); > > > + m_min_type= 256; > > > + m_max_type= 0; > > > + m_min_length= 256; > > > + m_max_length= 0; > > > + add(&Type_handler_inet6::s_singleton); > > Eh? I thought it's added in the Type_handler_inet6 constructor. I think in the end there should be two separate steps actually: - a constructor for Type_handler_xxx - a method to add it into the registry So the plugin loading routine will first create a handler from a dynamic library, and then put it into the registry. Now I just implemented it by a special "class Type_handler_inet6_singleton".

2 4

[Maria-developers] Unsubscribe
by Abdul Hakeem 28 Feb '16

28 Feb '16

1 0

[Maria-developers] win.test fails with --ps-protocol
by Sergey Petrunia 27 Feb '16

27 Feb '16

Hello Igor, I'm looking at the buildbot http://buildbot.askmonty.org/buildbot/grid?category=main&branch=bb-10.2-mde… and I see that your push https://github.com/MariaDB/server/commit/0c6d753708dfe07607ab6879b4832fe6ec… causes failures for win.test with --ps-protocol: http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/4911/st… Question #1: can you fix this? Question #2: Also, stacktrace printing is broken in the tree: mysqld: /home/buildbot/buildbot/build/mariadb-10.2.0/sql/set_var.cc:1205: sys_var::where get_sys_var_value_origin(void*): Assertion `!mysqld_server_started' failed. Fatal signal 6 while backtracing Could a failure like this be caused by "the backport" that was pushed earlier? BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] a849d1b: MDEV-9488 - Table cache cleanups
by Sergei Golubchik 24 Feb '16

24 Feb '16

Hi, Sergey! On Jan 29, Sergey Vojtovich wrote: > revision-id: a849d1bac80f2b38bc6bc170388e2f3cc1d4e1ad (mariadb-10.1.8-113-ga849d1b) > parent(s): 5f656d0dca8aae9486d91191ee6b6da6b9c552bf > committer: Sergey Vojtovich > timestamp: 2016-01-29 12:58:50 +0400 > message: > > MDEV-9488 - Table cache cleanups > > Remove tdc_acquire_share() helpers: they don't actually make things simpler. Hmm, okay. ok to push. Regards, Sergei

1 0

Re: [Maria-developers] eff48be: MDEV-9488 - Table cache cleanups
by Sergei Golubchik 24 Feb '16

24 Feb '16

Hi, Sergey! On Jan 29, Sergey Vojtovich wrote: > revision-id: eff48be5f1f9aedd2b0d6690d0883d05ef7f44a1 (mariadb-10.1.8-114-geff48be) > parent(s): a849d1bac80f2b38bc6bc170388e2f3cc1d4e1ad > committer: Sergey Vojtovich > timestamp: 2016-01-29 12:59:52 +0400 > message: > > MDEV-9488 - Table cache cleanups > > Cleanup now unused arguments of tdc_open_view(). ok to push Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 5f656d0: MDEV-9488 - Table cache cleanups
by Sergei Golubchik 24 Feb '16

24 Feb '16

Hi, Sergey! On Jan 29, Sergey Vojtovich wrote: > revision-id: 5f656d0dca8aae9486d91191ee6b6da6b9c552bf (mariadb-10.1.8-112-g5f656d0) > parent(s): 7b50447aa6d051b8d14bb01ef14802cb8ffee223 > committer: Sergey Vojtovich > timestamp: 2016-01-29 12:58:24 +0400 > message: > > MDEV-9488 - Table cache cleanups > > tdc_assign_new_table_id() does not relate to table cache, move it out of > table_cache.cc. ok to push Regards, Sergei

1 0

Re: [Maria-developers] 59e1aaa: MDEV-7331 - information_schema.user_variables
by Sergei Golubchik 24 Feb '16

24 Feb '16

Hi, Sergey! On Feb 24, Sergey Vojtovich wrote: > revision-id: 59e1aaa5f71cb725ce5242eb5abf2e00c6f5a793 (mariadb-10.1.8-124-g59e1aaa) > parent(s): 0485328d030f4b742dac7b667e8ed099beb9e9f2 > committer: Sergey Vojtovich > timestamp: 2016-02-24 14:31:43 +0400 > message: > > +static ST_FIELD_INFO user_variables_fields_info[] = > +{ > + { "VARIABLE_NAME", 64, MYSQL_TYPE_STRING, 0, 0, "Variable_name", 0 }, > + { "VARIABLE_VALUE", 2048, MYSQL_TYPE_STRING, 0, MY_I_S_MAYBE_NULL, "Value", 0 }, > + { 0, 0, MYSQL_TYPE_NULL, 0, 0, 0, 0 } > +}; 1. As discussed on irc, let's show everything we know about variables. Hmm, let me see what's in user_var_entry structure... The type, the character set, signed/unsigned. I think, query_id would be not very useful, though. See COLUMNS and SYSTEM_VARIABLES for appropriate column names. 2. You have defined old_name's for columns - "Variable_name" and "Value". This automatically enables SHOW USER_VARIABLES statement. See https://mariadb.com/kb/en/mariadb/information-schema-plugins-show-and-flush… You shouldn't enable SHOW if it displays exactly the same content as I_S.USER_VARIABLES. But if you'll add all the other metadata fields to the I_S table, then SHOW could be used to display a subset of the columns, e.g. only variable name and the value. Just as you have now :) 3. An interesting extension could be a FLUSH USER_VARIABLES statement that would empty the thd->user_vars hash. It's very easy to do, but I don't know if it'll be very useful :) See the url above for the API. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MDEV-7231 Field ROUTINE_DEFINITION in INFORMATION_SCHEMA.`ROUTINES` contains broken procedure body when used shielding quotes inside.
by Alexander Barkov 22 Feb '16

22 Feb '16

Hi Sergei, Please review a patch for MDEV-7231. Thanks.

2 4

[Maria-developers] Getting started
by Varun Gupta 19 Feb '16

19 Feb '16

Hey, I am Varun and this year I wish to take part in Google Summer of Code 2016 . I have been looking around the project ideas on the ideas page of MariaDB and am interested in the idea 'AGGREGATE STORE FUNCTIONS' . It would be helpful if someone could guide me where should i start looking to get more details related to this project idea. Regards

2 1

[Maria-developers] A proposal to deprecate syntax: SELECT a'test'
by Alexander Barkov 19 Feb '16

19 Feb '16

Hello Monty, all, We discussed this with Sergei and both think that it will be a good idea to deprecate this syntax: SELECT a'test'; where 'a' is an identifier and 'test' its alias. This is a non-standard way, and it conflicts with some other important standard SQL grammar. See below. Typified literals and a syntax conflict --------------------------------------- An identifier followed by a single-quoted text string is needed for typified literals: SELECT INET6'ffff::ffff'; We already support this for temporal literals, which is a part of the SQL standard: SELECT TIME'10:10:10'; SELECT TIMESTAMP'2001-01-01 10:20:30'; SELECT DATE'2001-01-01'; Note, the conflict already exists in MySQL and MariaDB. This script: DROP TABLE IF EXISTS t1; CREATE TABLE t1 (date date); INSERT INTO t1 VALUES ('2016-02-19'); SELECT date'2001-01-01' FROM t1; returns: +------------------+ | date'2001-01-01' | +------------------+ | 2001-01-01 | +------------------+ Notice, it returns a DATE literal, it does not return the value of the column t1.date using '2001-01-01' as an alias. Notice the difference: SELECT date`2001-01-01` FROM t1; +------------+ | 2001-01-01 | +------------+ | 2016-02-19 | +------------+ This is confusing. Shift-reduce conflicts ---------------------- By the way, the fact that we support single-quoted string as an alias is the reason for shift-reduce conflicts in sql_yacc.yy, because TIME/DATE/TIMESTAMP followed by 'string' can be interpreted in two ways: - a column name followed by an alias - an SQL-standard time literal What can be used instead ------------------------ We have a number of other ways to specify aliases: The standard ways: SELECT a AS test; SELECT a "test"; SELECT a AS "test"; MySQL/MariaDB extensions: SELECT a `test`; SELECT a AS `test`; That should be enough. Proposal -------- 1. In 10.1 we add a warning when a single-quoted string is used as a column alias. Something like this should work: Single quotes in a select sublist alias are deprecated. Use double quotes instead. 2. In 10.2 we disallow this syntax by default and add either sql_mode or old_mode to enable the old behavior. Sergei thinks that old_mode is something that should sooner or later be completely deleted. So perhaps sql_mode is better here, as the old behavior might be needed forever for MySQL compatibility. We'll appreciate your feedback. Thanks.

1 0

[Maria-developers] Google Summer of Code 2016
by Colin Charles 18 Feb '16

18 Feb '16

Hi! Its that time of year again and I believe that we plan to participate in Google Summer of Code 2016 again (we did so in 2013, 2014, and 2015). We are of course waiting to get accepted (we will know on Feb 29 2016) but that should not stop you, a *potential mentor* from adding your idea to the idea’s list page: https://mariadb.com/kb/en/mariadb/google-summer-of-code-2016/ Potential mentors should also see if there are Jira tasks they’d like completed: https://mariadb.atlassian.net/issues/?jql=labels%3Dgsoc16 (add more by adding the gsoc16 label) *Students* — please note that we have *not* been accepted *yet*, so asking us if you can do a project now is not necessarily useful (once we are accepted, then yes, please propose your interest) Here’s to Google Summer of Code 2016! cheers, -colin -- Colin Charles, http://bytebot.net/blog/ twitter: @bytebot | skype: colincharles "First they ignore you, then they laugh at you, then they fight you, then you win." -- Mohandas Gandhi

1 0

Re: [Maria-developers] [Commits] fd8e846: MDEV-9564 - added s390x to lib64 INSTALL_LIBDIR handling
by Sergei Golubchik 18 Feb '16

18 Feb '16

Hi, Sergey! Ok to push. I didn't like it at first, as x86_64 is amd64 on FreeBSD, x64 on Windows, and I don't know what else can there be. For example, FreeBSD doesn't use /usr/lib64. But your change is for RPM packages only - so it's fine. On Feb 18, Sergey Vojtovich wrote: > revision-id: fd8e846a3b049903706267d58e6d8e61eea97df8 (mariadb-10.1.11-17-gfd8e846) > parent(s): db5b51fb7e3360d4ebc303bd67363bc8324e86b6 > committer: Sergey Vojtovich > timestamp: 2016-02-18 11:01:22 +0400 > message: > > MDEV-9564 - added s390x to lib64 INSTALL_LIBDIR handling > > Adjusted INSTALL_LIBDIR detection so that it is set to "lib64" on any 64bit > system (not only x86_64). New condition is insipired by GNUInstallDirs cmake > module. > > --- > cmake/install_layout.cmake | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/cmake/install_layout.cmake b/cmake/install_layout.cmake > index 757166b..d1a261f 100644 > --- a/cmake/install_layout.cmake > +++ b/cmake/install_layout.cmake > @@ -137,7 +137,7 @@ SET(INSTALL_SCRIPTDIR_RPM "bin") > SET(INSTALL_SYSCONFDIR_RPM "/etc") > SET(INSTALL_SYSCONF2DIR_RPM "/etc/my.cnf.d") > # > -IF(CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64") > +IF(CMAKE_SIZEOF_VOID_P EQUAL 8) > SET(INSTALL_LIBDIR_RPM "lib64") > SET(INSTALL_PLUGINDIR_RPM "lib64/mysql/plugin") > ELSE() Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] Window functions code, MDEV-9543 and MDEV-8646 are now merged
by Sergey Petrunia 15 Feb '16

15 Feb '16

Hello, Following the Thursday discussion, I've managed to re-base 10.2-window_simple branch onto the bb-10.2-mdev9543. The result is now pushed into bb-10.2-mdev9543. https://github.com/MariaDB/server/commits/bb-10.2-mdev9543 Since rebase happened, 10.2-window_simple tree should be abandoned. Let's try to avoid re-bases and "take everything and re-commit" steps from now on. BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] bb-10.2-mdev9543 is missing item_windowfunc.*
by Sergey Petrunia 12 Feb '16

12 Feb '16

Hello Igor, The tree at bb-10.2-mdev9543 is missing item_windowfunc.* files. Could you please add them there and push? BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] MDEV-9541 Selecting a time using a datetime fails
by Alexander Barkov 12 Feb '16

12 Feb '16

Hi Monty, Please have a look into the latest comments in MDEV-9541 on how PostgreSQL, SQL Server, Oracle 11i and MySQL-5.7 work when comparing temporal data types. There is no a general rule. Every vendor implements it in a different way. 1. PostgreSQL almost follows the standard, which disallows direct comparison of TIME and TIMESTAMP/DATETIME. The only exception, it allows to compare a TIME field to a string literal which looks like TIMESTAMP: SELECT * FROM t1 WHERE a='2001-01-01 10:20:30'; a | b ----------+--------------------- 10:20:30 | 2001-01-01 10:20:30 But strangely not the other way around! Comparing of a TIMESTAMP field to a string which looks like TIME does not work. I did not find an explanation for this. 2. SQL Server - Comparison of TIME + VARCHAR is done as TIME - Comparison of DATETIME + VARCHAR is done as DATETIME - Comparison of TIME and DATETIME is not allowed - Comparison of TIME and INT is not allowed - Comparison of DATETIME and INT is allowed (not sure how exactly it works though) - Temporal literals {t'10:20:30'} have data type VARCHAR, this is just a syntactic variation of a string literal '10:20:30', with no any additional difference. 3. Oracle 11i Oracle does not have the TIME data type. So mixture of TIME and TIMESTAMP is not possible. But comparison of TIMESTAMP and VARCHAR is always done as TIMESTAMP, no matter what kind expressions are involved (fields, functions, literals). 4. MySQL-5.7 It has a special code: for a TIME field compared to a non-DATE constant, the constant argument is converted to TIME. But as soon as you wrap the field say into COALESCE, or put a non-constant instead of a constant, this works in a different way. So, comparing of a TIME field to a string literal that looks like a TIMESTAMP/DATETIME works similar in PostgreSQL, SQL Server, MySQL-5.7. However, this looks only as a coincidence for this particular corner case. Perhaps we should not fix that in 10.0. It seems that MariaDB since 5.3 handles comparison of TIME in a more consistent way (less exceptions). Btw, why not to add a warning every time when an implicit non-standard type conversion happens and suggest an explicit CAST? This will encourage users to avoid doubtful constructions. Also, I noticed that the SQL Server documentation says that comparison of different data types is a subject to change in the future versions, and using this is not recommended. Perhaps we should also say something like this. I propose instead of adding a quick fix for this exception, to introduce a conception of "type strength" or "type derivation" in 10.2. The idea is simple: 1. Comparison is done according to the type of the strongest side. 2. Strengths(derivations) are as follows: - A field or an SP variable has the strongest type derivation, say 4. - Functions have derivation between field and typified constants, say 3. - Some functions that return input arguments (e.g. COALESCE, LEFT) can repeat derivations of their arguments. - A typified constant like TIME'10:20:30' has derivation 2. - A typeless constant like '10:20:30' has a very weak derivation, say 1. - An explicit NULL has the weakest derivation, say 0. 3. Only in case of equal strengths the result depends on the data types. Perhaps data types can have some sort of "minor strength", for conflict resolution. Say DATETIME/TIMESTAMP can have more minor strength over TIME or DATE. So a DATETIME field is compared to a TIME field as DATETIME rather than TIME. This will give some advantages: - Universal comparison rules for all data types - This idea will simplify adding new data types a lot. They will just follow the universal rule. Otherwise, new data types (say INET6), we'll have to have different execution paths with some exceptions and exceptions from exceptions :) - The problem mentioned in MDEV-9541 will be gone: it will compare as TIME just because it's a field compared to a typeless constant. - We'll remove a special code covering comparison of YEAR and LONGLONG fields. They will just work as before, but now because a field always wins a constant, according to the universal rules. - We'll probably remove some examples of a special code covering explicit NULLs (i.e. Item::NULL_ITEM). - A field compared to a constant will always be able to use indexes. Less chances to put the server into a heavy load because of a non-smart query executed. - The same results for a field compared to some expression, and for the same field wrapped into a hybrid function like COALESCE. - Hybrid functions with field arguments will also be able to use indexes in some cases. - Equal field propagation will be possible in more cases. - It should be easier to add a new SQL mode for the strict SQL standard compliant comparison style. I think it's good to have this option. Greetings.

1 0

[Maria-developers] MDEV-9371 select insert('a', 2, 1, 'b') doesn't return expected 'a'
by Alexander Barkov 10 Feb '16

10 Feb '16

Hello Sergei, Please review a fix for MDEV-9371. I'd suggest to push it to 10.0 instead of 5.5, as MySQL fixed this bug in 5.6 (not 5.5). Also, I think it's worthy to backport MDEV-9332 to 10.0 as well, because the patch for MDEV-9371 touches the test which we modified in MDEV-9332. Thanks!

2 2

Re: [Maria-developers] [Commits] 841c317: MDEV-7122: Assertion `0' failed in subselect_hash_sj_engine::init
by Sergey Petrunia 09 Feb '16

09 Feb '16

Hi Vicentiu, Please find some cosmetic feedback below. Ok to push after it is addressed. On Tue, Feb 09, 2016 at 04:02:58PM +0200, Vicentiu Ciorbaru wrote: > revision-id: 841c317d237453f3e6c253e2a884cad73622b59a (mariadb-5.5.47-33-g841c317) > parent(s): ad94790f468bacb2cd62d02d3a6e7f012ca72e65 > author: Vicențiu Ciorbaru > committer: Vicențiu Ciorbaru > timestamp: 2016-02-09 16:00:18 +0200 > message: > > MDEV-7122: Assertion `0' failed in subselect_hash_sj_engine::init > > The select mentioned in the bug attempted to create a temporary table > using the maria storage engine. The table needs to have primary keys such that > duplicates can be removed. Unfortunately this use case has a longer > than allowed key and the tmp table got created without a temporary key. > We must not allow materialization for the subquery if the total key > length and key parts is greater than what the storage engine supports. > > --- > mysql-test/r/subselect.result | 12 ++++++++++++ > mysql-test/r/subselect_no_mat.result | 12 ++++++++++++ > mysql-test/r/subselect_no_opts.result | 12 ++++++++++++ > mysql-test/r/subselect_no_scache.result | 12 ++++++++++++ > mysql-test/r/subselect_no_semijoin.result | 12 ++++++++++++ > mysql-test/t/subselect.test | 9 +++++++++ > sql/opt_subselect.cc | 10 ++++++++++ > sql/sql_select.cc | 25 +++++++++++++++++++++++++ > 8 files changed, 104 insertions(+) > > diff --git a/mysql-test/r/subselect.result b/mysql-test/r/subselect.result > index ba52fd9..27e8eaa 100644 > --- a/mysql-test/r/subselect.result > +++ b/mysql-test/r/subselect.result > @@ -7081,3 +7081,15 @@ sq > NULL > deallocate prepare stmt; > drop table t1,t2,t3,t4; > +# MDEV-7122 > +# Assertion `0' failed in subselect_hash_sj_engine::init > +# > +SET SESSION big_tables=1; Please unset the variable back at the end of this testcase. We don't want the rest of the test to run with non-standard settings. > +CREATE TABLE t1(a char(255) DEFAULT '', KEY(a(10))) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci; > +INSERT INTO t1 VALUES(0),(0),(0); > +SELECT * FROM t1 WHERE a IN(SELECT MIN(a) FROM t1); > +a > +0 > +0 > +0 > +DROP TABLE t1; > diff --git a/mysql-test/r/subselect_no_mat.result b/mysql-test/r/subselect_no_mat.result > index d8d5c4e..4727f9f 100644 > --- a/mysql-test/r/subselect_no_mat.result > +++ b/mysql-test/r/subselect_no_mat.result > @@ -7078,6 +7078,18 @@ sq > NULL > deallocate prepare stmt; > drop table t1,t2,t3,t4; > +# MDEV-7122 > +# Assertion `0' failed in subselect_hash_sj_engine::init > +# > +SET SESSION big_tables=1; > +CREATE TABLE t1(a char(255) DEFAULT '', KEY(a(10))) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci; > +INSERT INTO t1 VALUES(0),(0),(0); > +SELECT * FROM t1 WHERE a IN(SELECT MIN(a) FROM t1); > +a > +0 > +0 > +0 > +DROP TABLE t1; > set optimizer_switch=default; > select @@optimizer_switch like '%materialization=on%'; > @@optimizer_switch like '%materialization=on%' > diff --git a/mysql-test/r/subselect_no_opts.result b/mysql-test/r/subselect_no_opts.result > index 78908bc..7c542dd 100644 > --- a/mysql-test/r/subselect_no_opts.result > +++ b/mysql-test/r/subselect_no_opts.result > @@ -7076,4 +7076,16 @@ sq > NULL > deallocate prepare stmt; > drop table t1,t2,t3,t4; > +# MDEV-7122 > +# Assertion `0' failed in subselect_hash_sj_engine::init > +# > +SET SESSION big_tables=1; > +CREATE TABLE t1(a char(255) DEFAULT '', KEY(a(10))) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci; > +INSERT INTO t1 VALUES(0),(0),(0); > +SELECT * FROM t1 WHERE a IN(SELECT MIN(a) FROM t1); > +a > +0 > +0 > +0 > +DROP TABLE t1; > set @optimizer_switch_for_subselect_test=null; > diff --git a/mysql-test/r/subselect_no_scache.result b/mysql-test/r/subselect_no_scache.result > index d1de4b8..3b58b3c 100644 > --- a/mysql-test/r/subselect_no_scache.result > +++ b/mysql-test/r/subselect_no_scache.result > @@ -7087,6 +7087,18 @@ sq > NULL > deallocate prepare stmt; > drop table t1,t2,t3,t4; > +# MDEV-7122 > +# Assertion `0' failed in subselect_hash_sj_engine::init > +# > +SET SESSION big_tables=1; > +CREATE TABLE t1(a char(255) DEFAULT '', KEY(a(10))) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci; > +INSERT INTO t1 VALUES(0),(0),(0); > +SELECT * FROM t1 WHERE a IN(SELECT MIN(a) FROM t1); > +a > +0 > +0 > +0 > +DROP TABLE t1; > set optimizer_switch=default; > select @@optimizer_switch like '%subquery_cache=on%'; > @@optimizer_switch like '%subquery_cache=on%' > diff --git a/mysql-test/r/subselect_no_semijoin.result b/mysql-test/r/subselect_no_semijoin.result > index 524a5dd..9246ffb 100644 > --- a/mysql-test/r/subselect_no_semijoin.result > +++ b/mysql-test/r/subselect_no_semijoin.result > @@ -7076,5 +7076,17 @@ sq > NULL > deallocate prepare stmt; > drop table t1,t2,t3,t4; > +# MDEV-7122 > +# Assertion `0' failed in subselect_hash_sj_engine::init > +# > +SET SESSION big_tables=1; > +CREATE TABLE t1(a char(255) DEFAULT '', KEY(a(10))) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci; > +INSERT INTO t1 VALUES(0),(0),(0); > +SELECT * FROM t1 WHERE a IN(SELECT MIN(a) FROM t1); > +a > +0 > +0 > +0 > +DROP TABLE t1; > set @optimizer_switch_for_subselect_test=null; > set @join_cache_level_for_subselect_test=NULL; > diff --git a/mysql-test/t/subselect.test b/mysql-test/t/subselect.test > index 67eac0d..5ca21ea 100644 > --- a/mysql-test/t/subselect.test > +++ b/mysql-test/t/subselect.test > @@ -5964,3 +5964,12 @@ EXECUTE stmt; > > deallocate prepare stmt; > drop table t1,t2,t3,t4; > + > +--echo # MDEV-7122 > +--echo # Assertion `0' failed in subselect_hash_sj_engine::init > +--echo # > +SET SESSION big_tables=1; > +CREATE TABLE t1(a char(255) DEFAULT '', KEY(a(10))) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci; > +INSERT INTO t1 VALUES(0),(0),(0); > +SELECT * FROM t1 WHERE a IN(SELECT MIN(a) FROM t1); > +DROP TABLE t1; > diff --git a/sql/opt_subselect.cc b/sql/opt_subselect.cc > index 827290f..a39e04b 100644 > --- a/sql/opt_subselect.cc > +++ b/sql/opt_subselect.cc > @@ -830,12 +830,14 @@ bool subquery_types_allow_materialization(Item_in_subselect *in_subs) > in_subs->sjm_scan_allowed= FALSE; > > bool all_are_fields= TRUE; > + uint32 total_key_length = 0; > for (uint i= 0; i < elements; i++) > { > Item *outer= in_subs->left_expr->element_index(i); > Item *inner= it++; > all_are_fields &= (outer->real_item()->type() == Item::FIELD_ITEM && > inner->real_item()->type() == Item::FIELD_ITEM); > + total_key_length += inner->max_length; > if (outer->cmp_type() != inner->cmp_type()) > DBUG_RETURN(FALSE); > switch (outer->cmp_type()) { > @@ -866,6 +868,14 @@ bool subquery_types_allow_materialization(Item_in_subselect *in_subs) > } > } > > + /* > + Make sure that create_tmp_table will not fail due to too long keys. > + See MDEV-7122. This check is performed inside create_tmp_table also and > + we must do it so that we know the table has keys created. > + */ > + if (total_key_length > HA_MAX_KEY_LENGTH || elements > HA_MAX_KEY_SEG) > + DBUG_RETURN(FALSE); > + > in_subs->types_allow_materialization= TRUE; > in_subs->sjm_scan_allowed= all_are_fields; > DBUG_PRINT("info",("subquery_types_allow_materialization: ok, allowed")); > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index f43f506..218a4ce 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -15307,6 +15307,13 @@ create_tmp_table(THD *thd, TMP_TABLE_PARAM *param, List<Item> &fields, > if (!table->file) > goto err; > > + /* > + Temporary table storage engines must allow keys of at least > + HA_MAX_KEY_LENGT and at least HA_MAX_KEY_SEG key parts. psergey: typo. ^^^^^^^^^^^ > + */ > + DBUG_ASSERT(table->file->max_key_length() >= HA_MAX_KEY_LENGTH && > + table->file->max_key_parts() >= HA_MAX_KEY_SEG); > + > if (!using_unique_constraint) > reclength+= group_null_items; // null flag is stored separately > > @@ -15940,6 +15947,12 @@ bool create_internal_tmp_table(TABLE *table, KEY *keyinfo, > goto err; > > bzero(seg, sizeof(*seg) * keyinfo->key_parts); > + /* > + Note that a simillar check is performed during typo ^^^^^^^^^ > + subquery_types_allow_materialization. See MDEV-7122 for more details as > + to why. Whenever this changes, it must be updated there as well, for > + all tmp_table engines. > + */ > if (keyinfo->key_length > table->file->max_key_length() || > keyinfo->key_parts > table->file->max_key_parts() || > share->uniques) > @@ -16126,6 +16139,12 @@ bool create_internal_tmp_table(TABLE *table, KEY *keyinfo, > goto err; > > bzero(seg, sizeof(*seg) * keyinfo->key_parts); > + /* > + Note that a simillar check is performed during typo ^^^^^^^^^ > + subquery_types_allow_materialization. See MDEV-7122 for more details as > + to why. Whenever this changes, it must be updated there as well, for > + all tmp_table engines. > + */ > if (keyinfo->key_length > table->file->max_key_length() || > keyinfo->key_parts > table->file->max_key_parts() || > share->uniques) > @@ -16286,6 +16305,12 @@ create_internal_tmp_table_from_heap2(THD *thd, TABLE *table, > if (!(new_table.file= get_new_handler(&share, &new_table.mem_root, > new_table.s->db_type()))) > DBUG_RETURN(1); // End of memory > + /* > + Temporary table storage engines must allow keys of at least > + HA_MAX_KEY_LENGTH and at least HA_MAX_KEY_SEG key parts. > + */ > + DBUG_ASSERT(new_table.file->max_key_length() >= HA_MAX_KEY_LENGTH && > + new_table.file->max_key_parts() >= HA_MAX_KEY_SEG); > > save_proc_info=thd->proc_info; > thd_proc_info(thd, proc_info); > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] MTR related community contribution
by Vicențiu Ciorbaru 07 Feb '16

07 Feb '16

Hi Vlad, I'm looking at the community pull request https://github.com/grooverdan/mariadb-server/commit/a9dfee63bad7393002621f9… I have a question regarding your commit: 2943d2b7e99b4221ef481890fc9a56ad569f3985 The contribution aims to remove the hard limit of 50 unique ids as it seems to cause problems for some use cases with a lot of proceeses. It does this by either leaving it at 50 or to be equal to the number of spawned processes. I'm thinking that even setting the limit of unique ids to num_processes might cause problems if there is another mtr running in parallel and both have > 50 processes. This is probably minor but why set this limit so low? Wouldn't a larger number (say 10k) be ok? Or why not generate a random number and lock a file with that number? (if the random number is invalid, generate a new one. On average it should finish quite quickly) Thanks, Vicentiu

3 2

[Maria-developers] Invalid code in server_audit.c
by Vicențiu Ciorbaru 06 Feb '16

06 Feb '16

Hi Alexei, While investigating community contributions from Daniel Black (MDEV-9433), I came across this comment: [image: danblack]Daniel Black <https://mariadb.atlassian.net/secure/ViewProfile.jspa?name=danblack> added a comment - 2016-01-20 02:54 [plugin/server_audit/server_audit.c:402]: (error) Invalid number of character '{' when these macros are defined: 'DBUG_VOID_RETURN=return;DBUG_RETURN(a)=return a;HAVE_PSI_INTERFACE'. [plugin/server_audit/server_audit.c:402]: (error) Invalid number of character '{' when these macros are defined: 'DBUG_VOID_RETURN=return;DBUG_RETURN(a)=return a;HAVE_PSI_INTERFACE;HUGETLB_USE_PROC_MEMINFO'. [plugin/server_audit/server_audit.c:402]: (error) Invalid number of character '{' when these macros are defined: 'DBUG_VOID_RETURN=return;DBUG_RETURN(a)=return a;HAVE_PSI_INTERFACE;USE_ALARM_THREAD'. Exists in 5.5 https://github.com/MariaDB/server/blob/5.5/plugin/server_audit/server_audit… Remove second {{ I've had a look at the code mentioned there and key_LOCK_bigbuffer does not exist anywhere else. Since we're not getting any compilation issues, it means we're not compiling that bit of code due to the defines. This indeed is so, as MYSQL_DYNAMIC_PLUGIN is defined for that file on the default cmake config which leads to FLOGGER being defined. I could not figure out a way to actually compile the code with defines such that that bit of code actually compiles. Either way it would not compile, due to the missing symbol. Do you think that we should just remove the line that mentions key_LOCK_bigbuffer? Also, since this is something we have not caught during testing, can we test for this compilation configuration somehow? Vicentiu

1 0

[Maria-developers] MariaDB JIRA is moving
by Rasmus Johansson 05 Feb '16

05 Feb '16

The MariaDB JIRA instance that currently is in use for project and issue tracking will change. The current instance is hosted in Atlassian's cloud and it has worked well, but we have hit the maximum user limit of 2000 users. It's fantastic to see how many of you actually report bugs and other issues in the MariaDB project! To hit that limit also means that we have to migrate over to a self-hosted instance of JIRA. Below are important details about that change. When will the switch happen: Sat 27th of Feb What is the impact for me as a user of MariaDB's JIRA: 1. URL change - The current instance is hosted in Atalassian's cloud and has the URL mariadb.atlassian.net. The new one will be found at jira.mariadb.org starting from 28th of Feb. 2. RSS feeds - If you have been using JIRA's RSS feeds, you'll need to set up them again since the URL is changing What will happen to old JIRA: The mariadb.atlassian.net JIRA instance will be put into read-only mode and a banner will be visible on each page telling the URL of the new MariaDB JIRA instance. It will remain this way for some time since it has been referenced quite heavily from other sites. source: https://mariadb.org/mariadb-jira-is-moving/ -- Rasmus Johansson, VP Engineering MariaDB | t: +358 50 499 9589 | Skype: ratzpo

1 0

[Maria-developers] MDEV-8691 Multiple test failures on mips and mipsel platforms, suites federated and rpl
by Alexander Barkov 02 Feb '16

02 Feb '16

Hi Jan, can you please have a look into: https://mariadb.atlassian.net/browse/MDEV-8691 Otto managed to get a stack trace from mips machine. It seems to crash inside InnoDB. Thanks. #3 <signal handler called> #4 0x000000fff6b48e80 in std::__detail::_List_node_base::_M_hook(std::__detail::_List_node_base*) () from /usr/lib/mips64el-linux-gnuabi64/libstdc++.so.6 #5 0x00000001207cf878 in std::__cxx11::list<unsigned char*, std::allocator<unsigned char*> >::_M_insert (__x=<synthetic pointer>, __position=..., this=0x118) at /usr/include/c++/5/bits/stl_list.h:1755 #6 std::__cxx11::list<unsigned char*, std::allocator<unsigned char*> >::push_back (__x=<synthetic pointer>, this=0x118) at /usr/include/c++/5/bits/stl_list.h:1089 #7 recv_dblwr_t::add (this=0x118, page=0xffeb004000 "\377\060~$") at /«PKGBUILDDIR»/storage/xtradb/log/log0recv.cc:4077 #8 0x00000001208df7f8 in buf_dblwr_init_or_load_pages (file=<optimized out>, path=0xfff678b029 <srv_read_only_mode> "", load_corrupt_pages=<optimized out>) at /«PKGBUILDDIR»/storage/xtradb/buf/buf0dblwr.cc:457 #9 0x000000fff66e42bc in open_or_create_data_files (sum_of_new_sizes=<synthetic pointer>, max_flushed_lsn=0xffffa89a50, min_flushed_lsn=0xffffa89a48, create_new_db=<synthetic pointer>) at /«PKGBUILDDIR»/storage/innobase/srv/srv0start.cc:1033 #10 innobase_start_or_create_for_mysql () at /«PKGBUILDDIR»/storage/innobase/srv/srv0start.cc:2105 #11 0x000000fff65f0028 in innobase_init (p=<optimized out>) at /«PKGBUILDDIR»/storage/innobase/handler/ha_innodb.cc:3400 #12 0x00000001204a17bc in ha_initialize_handlerton (plugin=0xfff5cb49f0) at /«PKGBUILDDIR»/sql/handler.cc:501 #13 0x00000001202fcafc in plugin_initialize (tmp_root=0xffffa91c40, tmp_root@entry=0x0, plugin=plugin@entry=0xfff5cb49f0, argc=0x120fdf680 <remaining_argc>, argc@entry=0x6, argv=0xfff5c7cc58, argv@entry=0xfd8, options_only=<optimized out>) at /«PKGBUILDDIR»/sql/sql_plugin.cc:1379 #14 0x00000001202fd7b0 in plugin_init (argc=0x6, argv=0xfd8, flags=<optimized out>) at /«PKGBUILDDIR»/sql/sql_plugin.cc:1659 #15 0x00000001202347ec in init_server_components () at /«PKGBUILDDIR»/sql/mysqld.cc:4850 #16 0x000000012023b83c in mysqld_main (argc=255, argv=0xff00000000) at /«PKGBUILDDIR»/sql/mysqld.cc:5445 #17 0x000000012021cf18 in main (argc=<optimized out>, argv=<optimized out>) at /«PKGBUILDDIR»/sql/main.cc:25

2 2

Re: [Maria-developers] 9f2e2d8: [MDEV-7978] Update grammar for new syntax
by Sergei Golubchik 02 Feb '16

02 Feb '16

Hi, Vicentiu! On Jan 19, Vicentiu Ciorbaru wrote: > revision-id: 9f2e2d89917709f9223fe2d4b75dc86d0297b4c4 (mariadb-10.1.8-113-g9f2e2d8) > parent(s): 83e197aa6bd3a7eb81482795732bfbdb3b415deb > author: Vicențiu Ciorbaru > committer: Vicențiu Ciorbaru > timestamp: 2016-01-17 17:07:21 +0200 > message: > > [MDEV-7978] Update grammar for new syntax > > Extend the syntax accepted by the grammar to account for the new create user > and alter user syntax. a couple of comments: > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index 3eb6a99..c1f4249 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -7279,6 +7280,11 @@ alter: > lex->sql_command= SQLCOM_ALTER_SERVER; > lex->server_options.reset($3); > } OPTIONS_SYM '(' server_options_list ')' { } > + | ALTER opt_if_exists USER clear_privileges user_list > + require_clause resource_options > + { please, add a comment that "ALTER USER foo;" is allowed for MySQL compatibility > + Lex->sql_command= SQLCOM_ALTER_USER; > + } > ; > > ev_alter_on_schedule_completion: > @@ -15734,6 +15724,37 @@ grant_option: > } > ; > > +resource_option_list: > + resource_option_list resource_option {} > + | resource_option {} > + ; > + > +resource_options: better rename it to opt_resource_options (and opt_grant_options below) for consistency. In sql_yacc.yy optional clauses often start with opt_ and one can immediately see that they're optional in, for example | ALTER opt_if_exists USER clear_privileges user_list opt_require_clause opt_resource_options > + /* empty */ {} > + | WITH resource_option_list > + ; > + > + > +grant_options: > + /* empty */ {} > + | WITH grant_option_list {} > + ; Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MDEV-9415 Crash with ConnectSE + MySQL ODBC driver + INSERT
by Alexander Barkov 02 Feb '16

02 Feb '16

Hi Georg, Can you please have a look into: https://mariadb.atlassian.net/browse/MDEV-9415 ConnectSE crashes on INSERT statements inside the ODBC driver, while UPDATE, SELECT, DELETE work fine. Any ideas how to track down the problem? Thanks.

1 1

Re: [Maria-developers] [Commits] 5dd6eb4: MDEV-9117: Client Server capability negotiation for MariaDB specific functionality
by Sergei Golubchik 01 Feb '16

01 Feb '16

Hi, Sanja! On Jan 28, OleksandrByelkin wrote: > revision-id: 5dd6eb45c418f61c749a7884d7b3e13d62f3bfb9 (mariadb-10.1.8-113-g5dd6eb4) > parent(s): 14d79a5e99f9992cabb7489203989ccdaacadb78 > committer: Oleksandr Byelkin > timestamp: 2016-01-28 14:58:12 +0100 > message: > > MDEV-9117: Client Server capability negotiation for MariaDB specific functionality Just a couple of comments: > New capability flags space. > Removed old progress flag, added new one. > > diff --git a/client/mysql.cc b/client/mysql.cc > index 3e08c87..715eed8 100644 > --- a/client/mysql.cc > +++ b/client/mysql.cc > @@ -152,7 +152,7 @@ static ulong opt_max_allowed_packet, opt_net_buffer_length; > static uint verbose=0,opt_silent=0,opt_mysql_port=0, opt_local_infile=0; > static uint my_end_arg; > static char * opt_mysql_unix_port=0; > -static int connect_flag=CLIENT_INTERACTIVE; > +static ulong connect_flag=CLIENT_INTERACTIVE; ulong width is not portable, remember trubles we had with ulong sysvars? better use ulonglong for 64bit and uint for 32bit. > static my_bool opt_binary_mode= FALSE; > static int interrupted_query= 0; > static char *current_host,*current_db,*current_user=0,*opt_password=0, > diff --git a/libmysql/client_settings.h b/libmysql/client_settings.h > index b233614..2577870 100644 > --- a/libmysql/client_settings.h > +++ b/libmysql/client_settings.h > @@ -27,7 +27,7 @@ extern char * mysql_unix_port; > When adding capabilities here, consider if they should be also added to > the server's version. > */ > -#define CLIENT_CAPABILITIES (CLIENT_LONG_PASSWORD | \ > +#define CLIENT_CAPABILITIES (CLIENT_MYSQL | \ really? CLIENT_MYSQL? Is it because they aren't using Connector/C yet? > CLIENT_LONG_FLAG | \ > CLIENT_TRANSACTIONS | \ > CLIENT_PROTOCOL_41 | \ > diff --git a/sql/sql_acl.cc b/sql/sql_acl.cc > index 2bcd351..dcb12e6 100644 > --- a/sql/sql_acl.cc > +++ b/sql/sql_acl.cc > @@ -11730,18 +11731,27 @@ static ulong parse_client_handshake_packet(MPVIO_EXT *mpvio, > */ > DBUG_ASSERT(net->read_pos[pkt_len] == 0); > > - ulong client_capabilities= uint2korr(net->read_pos); > + ulonglong client_capabilities= uint2korr(net->read_pos); > + compile_time_assert(sizeof(client_capabilities) >= 8); > if (client_capabilities & CLIENT_PROTOCOL_41) > { > - if (pkt_len < 4) > + if (pkt_len < 32) > return packet_error; > client_capabilities|= ((ulong) uint2korr(net->read_pos+2)) << 16; > + if (!(client_capabilities & CLIENT_MYSQL)) > + { > + // it is client with mariadb extensions > + ulonglong ext_client_capabilities= > + (((ulonglong)uint4korr(net->read_pos + 28)) << 32); > + client_capabilities|= ext_client_capabilities; > + } As I wrote earlier, please add here else if (client_capabilities & CLIENT_PROGRESS_OBSOLETE) client_capabilities|= MARIADB_CLIENT_PROGRESS; > } > > /* Disable those bits which are not supported by the client. */ > + compile_time_assert(sizeof(thd->client_capabilities) >= 8); > thd->client_capabilities&= client_capabilities; > > - DBUG_PRINT("info", ("client capabilities: %lu", thd->client_capabilities)); > + DBUG_PRINT("info", ("client capabilities 1: %llu", thd->client_capabilities)); why " 1" ? > if (thd->client_capabilities & CLIENT_SSL) > { > unsigned long errptr __attribute__((unused)); Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

2 1

[Maria-developers] Error creating new list
by Sonali Ellath 01 Feb '16

01 Feb '16

Hi, I built mailman-bundler using the guidelines found here <https://gitlab.com/mailman/mailman-bundler>. Server started successfully and and I created few domains. It worked well but when I tried to create a list, I received HTTP Error 500: A server error occurred. Please contact the administrator. Traceback: Traceback (most recent call last): File "/usr/lib/python3.4/wsgiref/handlers.py", line 137, in run self.result = application(self.environ, self.start_response) File "github/mailman/mailman-bundler/venv-3.4/lib/python3.4/site-packages/mailman/database/transaction.py", line 57, in wrapper rtn = function(*args, **kws) File "github/mailman/mailman-bundler/venv-3.4/lib/python3.4/site-packages/mailman/rest/wsgiapp.py", line 65, in __call__ environ, start_response) File "github/mailman/mailman-bundler/venv-3.4/lib/python3.4/site-packages/falcon/api.py", line 182, in __call__ responder(req, resp, **params) File "github/mailman/mailman-bundler/venv-3.4/lib/python3.4/site-packages/mailman/rest/lists.py", line 206, in on_post mlist = create_list(**validator(request)) File "github/mailman/mailman-bundler/venv-3.4/lib/python3.4/site-packages/mailman/app/lifecycle.py", line 81, in create_list call_name(config.mta.incoming).create(mlist) File "github/mailman/mailman-bundler/venv-3.4/lib/python3.4/site-packages/mailman/mta/postfix.py", line 69, in create self.regenerate() File "github/mailman/mailman-bundler/venv-3.4/lib/python3.4/site-packages/mailman/mta/postfix.py", line 104, in regenerate raise RuntimeError(NL.join(errors)) RuntimeError: command failure: /usr/sbin/postmap github/mailman/mailman-bundler/var/data/postfix_lmtp, 127, Key has expired command failure: /usr/sbin/postmap github/mailman/mailman-bundler/var/data/postfix_domains, 127, Key has expired Domain Details: Mail Host: lists.example.com URL Host: http://lists.example.com Please Help! Thanks - Sonali

2 1

[Maria-developers] Non-bloking API problem on non-x86 (MDEV-9112)
by Kristian Nielsen 01 Feb '16

01 Feb '16

Hi Georg, I stumbled upon the problem in the server source tree that ucontext check is broken, causing the non-blocking library to be completely disabled on all non-x86 platforms. After creating my own patch, I noticed that you alrady made a patch for this in MDEV-9112, but it does not appear to be pushed yet. Do you still want to push your patch? The patch looks fine (you can take this as review), it handles a couple things I forgot in my own patch. I think this should definitely go in 10.0 (and 10.1) - but do you think it should go in 5.5 also (not sure)? - Kristian. PS. I supplied a patch in CONC-153 for the similar problem in mariadb-connector-c, but maybe you are already aware of that as well. It was BTW cool to see that an application of mine (originally written against libmysqlclient) worked without changes on mariadb-connector-c.

1 1

Re: [Maria-developers] [Commits] 2a181ea: MDEV-8713 Add continuous binary log backup to mysqlbinlog.
by Sergei Golubchik 31 Jan '16

31 Jan '16

Hi, Alexey! On Nov 27, Alexey Botchkov wrote: > revision-id: 2a181ea7b16dcbc6dd30af5a2396760261d7d500 (mariadb-10.1.8-81-g2a181ea) > parent(s): 9d5c9379a6fd3662d4388cda177a3b1ea7b070cc > committer: Alexey Botchkov > timestamp: 2015-11-27 16:33:23 +0400 > message: > > MDEV-8713 Add continuous binary log backup to mysqlbinlog. > diff --git a/client/client_priv.h b/client/client_priv.h > index 656c8fc..44f0b21 100644 > --- a/client/client_priv.h > +++ b/client/client_priv.h > @@ -92,6 +92,7 @@ enum options_client > OPT_REPORT_PROGRESS, > OPT_SKIP_ANNOTATE_ROWS_EVENTS, > OPT_SSL_CRL, OPT_SSL_CRLPATH, > + OPT_RAW_OUTPUT, OPT_WAIT_SERVER_ID, OPT_STOP_NEVER, 1. There's no need to create named constants for OPT_RAW_OUTPUT and OPT_WAIT_SERVER_ID - they are never used anywhere. Simply put 0 in the my_options structure instead. 2. OPT_STOP_NEVER is used in the switch, so it needs a named constant. but it still doesn't have to be in client_priv.h, it's not an option common to all clients. Define it in mysqlbinlog.cc (see how mysqltest.cc and mysql_upgrade.c do it) > OPT_MAX_CLIENT_OPTION /* should be always the last */ > }; > > diff --git a/client/mysqlbinlog.cc b/client/mysqlbinlog.cc > index 5b07232..a2509b7 100644 > --- a/client/mysqlbinlog.cc > +++ b/client/mysqlbinlog.cc > @@ -96,6 +98,8 @@ static char* database= 0; > static my_bool force_opt= 0, short_form= 0, remote_opt= 0; > static my_bool debug_info_flag, debug_check_flag; > static my_bool force_if_open_opt= 1; > +static my_bool opt_raw_mode= 0, opt_stop_never= 0; > +static int64 opt_stop_never_slave_server_id= -1; longlong, not int64. Because 1) you've used GET_LL for it, and 2) we generally don't use int64 (it's only used in special cases to mean exactly "signed 64-bit integer"). > static my_bool opt_verify_binlog_checksum= 1; > static ulonglong offset = 0; > static char* host = 0; > @@ -1368,8 +1373,14 @@ static struct my_option my_options[] = > {"read-from-remote-server", 'R', "Read binary logs from a MySQL server.", > &remote_opt, &remote_opt, 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, > 0, 0}, > - {"result-file", 'r', "Direct output to a given file.", 0, 0, 0, GET_STR, > - REQUIRED_ARG, 0, 0, 0, 0, 0, 0}, > + {"raw", OPT_RAW_OUTPUT, "Requires -R. Output raw binlog data instead of SQL " > + "statements and write it to files corresponding to server logs.", What does that mean? "write it to files corresponding to server logs"? Perhaps you could clarify the help text? > + &opt_raw_mode, &opt_raw_mode, 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, > + 0, 0}, > + {"result-file", 'r', "Direct output to a given file. With --raw this is a " > + "prefix for the file names.", > + &result_file_name, &result_file_name, 0, GET_STR, REQUIRED_ARG, > + 0, 0, 0, 0, 0, 0}, > {"server-id", 0, > "Extract only binlog entries created by the server having the given id.", > &server_id, &server_id, 0, GET_ULONG, > @@ -1418,6 +1429,16 @@ static struct my_option my_options[] = > "(you should probably use quotes for your shell to set it properly).", > &stop_datetime_str, &stop_datetime_str, > 0, GET_STR_ALLOC, REQUIRED_ARG, 0, 0, 0, 0, 0, 0}, > + {"stop-never", OPT_STOP_NEVER, "Wait for more data from the server " > + "instead of stopping at the end of the last log. Implicitly sets " > + "--to-last-log but instead of stopping at the end of the last log " > + "it continues to wait till the server disconnects.", I don't know why you needed to say the same thing twice. What about: Wait for more data from the server instead of stopping at the end of the last log. Implies --to-last-log. > + &opt_stop_never, &opt_stop_never, 0, > + GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0}, > + {"stop-never-slave-server-id", OPT_WAIT_SERVER_ID, > + "The slave server_id used for --read-from-remote-server --stop-never.", > + &opt_stop_never_slave_server_id, &opt_stop_never_slave_server_id, 0, > + GET_LL, REQUIRED_ARG, -1, -1, 0xFFFFFFFFLL, 0, 0, 0}, GET_LL, really? So, negative server_id's are allowed? Either do it like the existing --server-id option (GET_ULONG, 0 means "not set"), or, if you want to allow 0 as a valid server-id value, you can do, like case OPT_WAIT_SERVER_ID: if (argument == disabled_my_option) opt_stop_never_slave_server_id= -1; break; but really I wouldn't bother and just do like --server-id does. > {"stop-position", OPT_STOP_POSITION, > "Stop reading the binlog at position N. Applies to the last binlog " > "passed on the command line.", > @@ -1701,6 +1718,9 @@ get_one_option(int optid, const struct my_option *opt __attribute__((unused)), > print_version(); > opt_version= 1; > break; > + case OPT_STOP_NEVER: > + to_last_remote_log= 1; > + break; This is wrong, it allows someone to do mysqlbinlog --stop-never --to-last-remote-log=0 and you get incorrect configuration. Alternatively one can do mysqlbinlog --stop-never=1 --stop-never=0 and to_last_remote_log will be magically enabled. Heck, one can even do mysqlbinlog --stop-never=0 and that'll enable to_last_remote_log. The correct approach is to do if (opt_stop_never) to_last_remote_log= TRUE; and only *after* all options are read. Note that in this case you won't need OPT_STOP_NEVER constant either (see my comment earlier in this review) > case '?': > usage(); > opt_version= 1; > @@ -1922,6 +1943,240 @@ static Exit_status check_master_version() > } > > + > + > +static char out_file_name[FN_REFLEN + 1]; > + > +static Exit_status handle_event_raw_mode(PRINT_EVENT_INFO *print_event_info, > + ulong *len, > + const char* logname, uint logname_len) > +{ I'm not reviewing the logic of handle_event_raw_mode, as far as I can see you've copied it from mysqlbinlog in MySQL. Did you use the latest version of mysqlbinlog or the one from the original patch? Use the latest one, it might have bugfixes. > + const char *error_msg; > + const unsigned char *read_pos= mysql->net.read_pos + 1; > + Log_event_type type; > + DBUG_ENTER("handle_event_raw_mode"); > + DBUG_ASSERT(opt_raw_mode && remote_opt); > + > + type= (Log_event_type) read_pos[EVENT_TYPE_OFFSET]; > + > + if (type == HEARTBEAT_LOG_EVENT) > + DBUG_RETURN(OK_CONTINUE); > + > + if (type == ROTATE_EVENT || type == FORMAT_DESCRIPTION_EVENT) > + { > + Log_event *ev; > + if (!(ev= Log_event::read_log_event((const char*) read_pos , > + *len - 1, &error_msg, > + glob_description_event, > + opt_verify_binlog_checksum))) > + { > + error("Could not construct annotate event object: %s", error_msg); may be: "%s", type == ROTATE_EVENT ? "rotate":"annotate" > + DBUG_RETURN(ERROR_STOP); > + } > + /* > + If reading from a remote host, ensure the temp_buf for the > + Log_event class is pointing to the incoming stream. > + */ > + ev->register_temp_buf((char *) read_pos, FALSE); > + > @@ -2000,113 +2264,23 @@ static Exit_status dump_remote_log_entries(PRINT_EVENT_INFO *print_event_info, > break; // end of data > DBUG_PRINT("info",( "len: %lu net->read_pos[5]: %d\n", > len, net->read_pos[5])); > - if (net->read_pos[5] == ANNOTATE_ROWS_EVENT) > + if (opt_raw_mode) > { > - if (!(ev= read_remote_annotate_event(net->read_pos + 1, len - 1, > - &error_msg))) > - { > - error("Could not construct annotate event object: %s", error_msg); > - DBUG_RETURN(ERROR_STOP); > - } > + retval= handle_event_raw_mode(print_event_info, &len, > + logname, logname_len); > } > else > { > - if (!(ev= Log_event::read_log_event((const char*) net->read_pos + 1 , > - len - 1, &error_msg, > - glob_description_event, > - opt_verify_binlog_checksum))) > - { > - error("Could not construct log event object: %s", error_msg); > - DBUG_RETURN(ERROR_STOP); > - } > - /* > - If reading from a remote host, ensure the temp_buf for the > - Log_event class is pointing to the incoming stream. > - */ > - ev->register_temp_buf((char *) net->read_pos + 1, FALSE); > + retval= handle_event_text_mode(print_event_info, &len, > + logname, logname_len, old_off); > } > - > - Log_event_type type= ev->get_type_code(); > - if (glob_description_event->binlog_version >= 3 || > - (type != LOAD_EVENT && type != CREATE_FILE_EVENT)) > + if (retval != OK_CONTINUE) > { > - /* > - If this is a Rotate event, maybe it's the end of the requested binlog; > - in this case we are done (stop transfer). > - This is suitable for binlogs, not relay logs (but for now we don't read > - relay logs remotely because the server is not able to do that). If one > - day we read relay logs remotely, then we will have a problem with the > - detection below: relay logs contain Rotate events which are about the > - binlogs, so which would trigger the end-detection below. > - */ > - if (type == ROTATE_EVENT) > - { > - Rotate_log_event *rev= (Rotate_log_event *)ev; > - /* > - If this is a fake Rotate event, and not about our log, we can stop > - transfer. If this a real Rotate event (so it's not about our log, > - it's in our log describing the next log), we print it (because it's > - part of our log) and then we will stop when we receive the fake one > - soon. > - */ > - if (rev->when == 0) > - { > - if (!to_last_remote_log) > - { > - if ((rev->ident_len != logname_len) || > - memcmp(rev->new_log_ident, logname, logname_len)) > - { > - DBUG_RETURN(OK_CONTINUE); > - } > - /* > - Otherwise, this is a fake Rotate for our log, at the very > - beginning for sure. Skip it, because it was not in the original > - log. If we are running with to_last_remote_log, we print it, > - because it serves as a useful marker between binlogs then. > - */ > - continue; > - } > - len= 1; // fake Rotate, so don't increment old_off > - } > - } > - else if (type == FORMAT_DESCRIPTION_EVENT) > - { > - /* > - This could be an fake Format_description_log_event that server > - (5.0+) automatically sends to a slave on connect, before sending > - a first event at the requested position. If this is the case, > - don't increment old_off. Real Format_description_log_event always > - starts from BIN_LOG_HEADER_SIZE position. > - */ > - if (old_off != BIN_LOG_HEADER_SIZE) > - len= 1; // fake event, don't increment old_off > - } > - Exit_status retval= process_event(print_event_info, ev, old_off, logname); > - if (retval != OK_CONTINUE) > - DBUG_RETURN(retval); > + if (retval == OK_EOF) > + break; > + DBUG_RETURN(retval); > } > - else > - { > - Load_log_event *le= (Load_log_event*)ev; > - const char *old_fname= le->fname; > - uint old_len= le->fname_len; > - File file; > - Exit_status retval; > > - if ((file= load_processor.prepare_new_file_for_old_format(le,fname)) < 0) > - DBUG_RETURN(ERROR_STOP); > - > - retval= process_event(print_event_info, ev, old_off, logname); > - if (retval != OK_CONTINUE) > - { > - my_close(file,MYF(MY_WME)); > - DBUG_RETURN(retval); > - } > - retval= load_processor.load_old_format_file(net,old_fname,old_len,file); > - my_close(file,MYF(MY_WME)); > - if (retval != OK_CONTINUE) > - DBUG_RETURN(retval); > - } Hmm, when you moved all that code above into handle_event_text_mode(), you seem to have changed the behavior for to_last_remote_log. Was (comments removed for brevity): if (!to_last_remote_log) { if ((rev->ident_len != logname_len) || memcmp(rev->new_log_ident, logname, logname_len)) { DBUG_RETURN(OK_CONTINUE); } continue; } Now instead of 'continue' you return OK_CONTINUE from handle_event_text_mode() and it does not skip the code below (after the comment Let's adjust), like the old 'continue' did. > /* > Let's adjust offset for remote log as for local log to produce > similar text and to have --stop-position to work identically. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] Aliasing slave_parallel_workers/slave_parallel_threads
by Colin Charles 29 Jan '16

29 Jan '16

Hi Kristian, I am referencing an old thread from March 2014 here where you don’t have an objection, but maybe this has slipped: https://lists.launchpad.net/maria-developers/msg06923.html MySQL 5.7 has been released and I’m sure you realise that it uses the variable slave_parallel_workers (http://dev.mysql.com/doc/refman/5.7/en/replication-options-slave.html#sysva…) In MariaDB Server 10.1 we use slave_parallel_threads (https://mariadb.com/kb/en/mariadb/replication-and-binary-log-server-system-…) From a DBA standpoint, and also from people using both databases in testing/production, this is confusing and scripts now have to have unnecessary added logic for if MariaDB or else MySQL. It would be wise for us to alias this, so that its easier from a DBA usability standpoint IMHO I planned to open up a Jira request, but since this was discussed on the list before I thought I would ask again to revive an old thread. cheers, -colin -- Colin Charles, http://bytebot.net/blog/ twitter: @bytebot | skype: colincharles "First they ignore you, then they laugh at you, then they fight you, then you win." -- Mohandas Gandhi

4 6

Re: [Maria-developers] [Commits] 56ae997: [MDEV-7978] Added show create user implementation.
by Sergei Golubchik 29 Jan '16

29 Jan '16

Hi, Vicentiu! Below are my comments on the combined patch: > diff --git a/mysql-test/r/alter_user.result b/mysql-test/r/alter_user.result > new file mode 100644 > index 0000000..2acc249 > --- /dev/null > +++ b/mysql-test/r/alter_user.result > @@ -0,0 +1,79 @@ > +select * from mysql.user where user = 'root' and host = 'localhost'; > +Host User Password Select_priv Insert_priv Update_priv Delete_priv Create_priv Drop_priv Reload_priv Shutdown_priv Process_priv File_priv Grant_priv References_priv Index_priv Alter_priv Show_db_priv Super_priv Create_tmp_table_priv Lock_tables_priv Execute_priv Repl_slave_priv Repl_client_priv Create_view_priv Show_view_priv Create_routine_priv Alter_routine_priv Create_user_priv Event_priv Trigger_priv Create_tablespace_priv ssl_type ssl_cipher x509_issuer x509_subject max_questions max_updates max_connections max_user_connections plugin authentication_string password_expired is_role default_role max_statement_time > +localhost root Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y 0 0 0 0 N N 0.000000 > +# Test syntax > +# > +# These 2 selects should have no changes from the first one. > +alter user CURRENT_USER; I'm not sure this should be a valid statement. After all ALTER TABLE t1; is not valid. There must be something after the user name > +select * from mysql.user where user = 'root' and host = 'localhost'; > +Host User Password Select_priv Insert_priv Update_priv Delete_priv Create_priv Drop_priv Reload_priv Shutdown_priv Process_priv File_priv Grant_priv References_priv Index_priv Alter_priv Show_db_priv Super_priv Create_tmp_table_priv Lock_tables_priv Execute_priv Repl_slave_priv Repl_client_priv Create_view_priv Show_view_priv Create_routine_priv Alter_routine_priv Create_user_priv Event_priv Trigger_priv Create_tablespace_priv ssl_type ssl_cipher x509_issuer x509_subject max_questions max_updates max_connections max_user_connections plugin authentication_string password_expired is_role default_role max_statement_time > +localhost root Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y 0 0 0 0 N N 0.000000 > +alter user CURRENT_USER(); > +select * from mysql.user where user = 'root' and host = 'localhost'; > +Host User Password Select_priv Insert_priv Update_priv Delete_priv Create_priv Drop_priv Reload_priv Shutdown_priv Process_priv File_priv Grant_priv References_priv Index_priv Alter_priv Show_db_priv Super_priv Create_tmp_table_priv Lock_tables_priv Execute_priv Repl_slave_priv Repl_client_priv Create_view_priv Show_view_priv Create_routine_priv Alter_routine_priv Create_user_priv Event_priv Trigger_priv Create_tablespace_priv ssl_type ssl_cipher x509_issuer x509_subject max_questions max_updates max_connections max_user_connections plugin authentication_string password_expired is_role default_role max_statement_time > +localhost root Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y 0 0 0 0 N N 0.000000 > +create user foo; > +select * from mysql.user where user = 'foo'; > +Host User Password Select_priv Insert_priv Update_priv Delete_priv Create_priv Drop_priv Reload_priv Shutdown_priv Process_priv File_priv Grant_priv References_priv Index_priv Alter_priv Show_db_priv Super_priv Create_tmp_table_priv Lock_tables_priv Execute_priv Repl_slave_priv Repl_client_priv Create_view_priv Show_view_priv Create_routine_priv Alter_routine_priv Create_user_priv Event_priv Trigger_priv Create_tablespace_priv ssl_type ssl_cipher x509_issuer x509_subject max_questions max_updates max_connections max_user_connections plugin authentication_string password_expired is_role default_role max_statement_time > +% foo N N N N N N N N N N N N N N N N N N N N N N N N N N N N N 0 0 0 0 N N 0.000000 > diff --git a/mysql-test/t/alter_user.test b/mysql-test/t/alter_user.test > new file mode 100644 > index 0000000..3a3a7d7 > --- /dev/null > +++ b/mysql-test/t/alter_user.test > @@ -0,0 +1,71 @@ > +--source include/not_embedded.inc > +--enable_connect_log > + > + > +select * from mysql.user where user = 'root' and host = 'localhost'; > +--echo # Test syntax > +--echo # > +--echo # These 2 selects should have no changes from the first one. > +alter user CURRENT_USER; > +select * from mysql.user where user = 'root' and host = 'localhost'; > +alter user CURRENT_USER(); > +select * from mysql.user where user = 'root' and host = 'localhost'; > + > +create user foo; > +select * from mysql.user where user = 'foo'; > +alter user foo; > +select * from mysql.user where user = 'foo'; > + > +--echo # Test super privilege works correctly with a read only database. > +SET @start_read_only = @@global.read_only; > +SET GLOBAL read_only=1; > +grant create user on *.* to foo; > + > +--echo # Currently no super privileges. > +connect (a, localhost, foo); > +select @@global.read_only; > + > +--error ER_OPTION_PREVENTS_STATEMENT > +alter user foo; > + > +--echo # Grant super privilege to the user. > +connection default; > +grant super on *.* to foo; > + > +--echo # We now have super privilege. We should be able to run alter user. > +connect (b, localhost, foo); > +alter user foo; > + > +connection default; > +SET GLOBAL read_only = @start_read_only; > + > +--echo # Test inexistant user. > +--error ER_CANNOT_USER > +alter user boo; > +--echo #--warning ER_CANNOT_USER > +alter if exists user boo; > + > +--echo # Test SSL related altering. eh? I don't see any SSL related altering here > +alter user foo identified by 'something'; > +select * from mysql.user where user = 'foo'; > + > +alter user foo identified by 'something2'; > +select * from mysql.user where user = 'foo'; > + > +alter user foo identified by password '*88C89BE093D4ECF72D039F62EBB7477EA1FD4D63'; > +select * from mysql.user where user = 'foo'; > + > +alter user foo identified with 'somecoolplugin'; > +select * from mysql.user where user = 'foo'; > + > +alter user foo identified with 'somecoolplugin' using 'somecoolpassphrase'; > +select * from mysql.user where user = 'foo'; > + > +--echo # Test resource limits altering. > +alter user foo with MAX_QUERIES_PER_HOUR 10 > + MAX_UPDATES_PER_HOUR 20 > + MAX_CONNECTIONS_PER_HOUR 30 > + MAX_USER_CONNECTIONS 40; > +select * from mysql.user where user = 'foo'; > +drop user foo; > +--disable_connect_log > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index 96985aa..9fcd505 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -4635,7 +4640,11 @@ mysql_execute_command(THD *thd) > break; > /* Conditionally writes to binlog */ > WSREP_TO_ISOLATION_BEGIN(WSREP_MYSQL_DB, NULL, NULL) > - if (!(res= mysql_rename_user(thd, lex->users_list))) > + if (lex->sql_command == SQLCOM_ALTER_USER) > + res= mysql_alter_user(thd, lex->users_list); I don't think ALTER USER needs same privileges as RENAME USER. A user should be able to ALTER herself. We have added a check_alter_user() helper exactly for that, remember? :) > + else > + res= mysql_rename_user(thd, lex->users_list); > + if (!res) > my_ok(thd); > break; > } > diff --git a/sql/sql_acl.cc b/sql/sql_acl.cc > index a6c6859..1285165 100644 > --- a/sql/sql_acl.cc > +++ b/sql/sql_acl.cc > @@ -7710,6 +7798,78 @@ static bool print_grants_for_role(THD *thd, ACL_ROLE * role) > } > > > +bool mysql_show_create_user(THD *thd, LEX_USER *lex_user) > +{ > + const char *username = safe_str(lex_user->user.str); > + const char *hostname = safe_str(lex_user->host.str); > + Protocol *protocol= thd->protocol; > + bool error= false; > + ACL_USER *acl_user; > + DBUG_ENTER("mysql_show_create_user"); > + > + // Check if the command specifies a username or not. > + if (lex_user->user.str == current_user.str) > + { > + username= thd->security_ctx->priv_user; > + hostname= thd->security_ctx->priv_host; > + } > + > + char buff[1024]; //Show create user should not take more than 1024 bytes. > + String field_name(buff, sizeof(buff), system_charset_info); There's a helper for that: StringBuffer<1024> field_name; on the other hand... you don't use field_name anywhere. > + List<Item> field_list; > + strxmov(buff, "CREATE USER for ", username, "@", hostname, NullS); > + Item_string *field = new (thd->mem_root) Item_string_ascii(thd, "", 0); > + if (!field) > + { > + my_error(ER_OUTOFMEMORY, MYF(0)); > + DBUG_RETURN(true); > + } > + Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

Re: [Maria-developers] 57bf586: MDEV-8615: Assertion `m_cpp_buf <= begin_ptr && begin_ptr <= m_cpp_buf + m_buf_length' failed in Lex_input_stream::body_utf8_start
by Sergei Golubchik 27 Jan '16

27 Jan '16

Hi, Sanja! On Jan 27, OleksandrByelkin wrote: > revision-id: 57bf58668877552ee43f4ed86689b8bbb3821c9c (mariadb-10.1.10-23-g57bf586) > parent(s): 825f51d1aab51d363dc07ec9fe0829af33063883 > committer: Oleksandr Byelkin > timestamp: 2016-01-27 00:38:36 +0100 > message: > > MDEV-8615: Assertion `m_cpp_buf <= begin_ptr && begin_ptr <= m_cpp_buf + m_buf_length' failed in Lex_input_stream::body_utf8_start > > Nothing should be done before any keyword recognized. Good! Just two comments: 1. May be it's better to rename s/pop_cs_label/pop_sp_label/ and s/pop_cs_empty_label/pop_sp_empty_label/ ? first, pop_sp_empty_label applies to all SPs, not only to compound statements, and second, pop_cs_label does not apply to compound statements at all :) 2. Your process_control_statement_label() makes no sense here. There is no need to do an if() - because in both branches you always know whether you have a label or not. Please split process_control_statement_label() in two different functions (or two yacc rules, whatever). > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index 302c2fd..3c2d386 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -248,6 +248,36 @@ static bool maybe_start_compound_statement(THD *thd) > return 0; > } > > +static bool process_control_statement_label(THD *thd, LEX_STRING label) > +{ > + if (label.str != NULL) > + { > + sp_pcontext *ctx= thd->lex->spcont; > + sp_label *lab= ctx->find_label(label); > + > + if (lab) > + { > + my_error(ER_SP_LABEL_REDEFINE, MYF(0), label.str); > + return 1; > + } > + else > + { > + lab= thd->lex->spcont->push_label(thd, label, > + thd->lex->sphead->instructions()); > + lab->type= sp_label::ITERATION; > + } > + } > + else > + { > + if (maybe_start_compound_statement(thd)) > + return 1; > + /* Unlabeled controls get an empty label. */ > + thd->lex->spcont->push_label(thd, empty_lex_str, > + thd->lex->sphead->instructions()); > + } > + return 0; > +} > + > /** > Helper action for a case expression statement (the expr in 'CASE expr'). > This helper is used for 'searched' cases only. > @@ -4389,6 +4374,84 @@ sp_control_content: > } > ; > > +pop_cs_label: > + sp_opt_label > + { > + sp_label *lab; > + Lex->sphead->backpatch(lab= Lex->spcont->pop_label()); > + if ($1.str) > + { > + if (my_strcasecmp(system_charset_info, $1.str, > + lab->name.str) != 0) > + { > + my_error(ER_SP_LABEL_MISMATCH, MYF(0), $1.str); > + MYSQL_YYABORT; > + } > + } > + } > + ; > + > +pop_cs_empty_label: > + { > + sp_label *lab; > + Lex->sphead->backpatch(lab= Lex->spcont->pop_label()); > + DBUG_ASSERT(lab->name.length == 0); > + } > + ; > + > +sp_labeled_control: > + label_ident ':' LOOP_SYM > + { > + if (process_control_statement_label(thd, $1)) > + MYSQL_YYABORT; > + } > + loop_body pop_cs_label > + { } > + | label_ident ':' WHILE_SYM > + { > + if (process_control_statement_label(thd, $1)) > + MYSQL_YYABORT; > + Lex->sphead->reset_lex(thd); > + } > + while_body pop_cs_label > + { } > + | label_ident ':' REPEAT_SYM > + { > + if (process_control_statement_label(thd, $1)) > + MYSQL_YYABORT; > + } > + repeat_body pop_cs_label > + { } > + ; > + > +sp_unlabeled_control: > + LOOP_SYM > + { > + if (process_control_statement_label(thd, null_lex_str)) > + MYSQL_YYABORT; > + } > + loop_body > + pop_cs_empty_label > + { } > + | WHILE_SYM > + { > + if (process_control_statement_label(thd, null_lex_str)) > + MYSQL_YYABORT; > + Lex->sphead->reset_lex(thd); > + } > + while_body > + pop_cs_empty_label > + { } > + | REPEAT_SYM > + { > + if (process_control_statement_label(thd, null_lex_str)) > + MYSQL_YYABORT; > + } > + repeat_body > + pop_cs_empty_label > + { } > + ; > + > trg_action_time: > BEFORE_SYM > { Lex->trg_chistics.action_time= TRG_ACTION_BEFORE; } Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] MDEV-5513 merge audit api changes from 5.5.34.
by Alexey Botchkov 27 Jan '16

27 Jan '16

Hello Sergei, all. I didn't quite understand what exact changes in 5.5.34 are important to us, so in order to begin i'm going to mention shortly what is new in MySQL 5.7 audit plugin API. 1. New event classes added: MYSQL_AUDIT_PARSE_CLASS = 2, MYSQL_AUDIT_AUTHORIZATION_CLASS = 3, MYSQL_AUDIT_TABLE_ACCESS_CLASS = 4, MYSQL_AUDIT_GLOBAL_VARIABLE_CLASS = 5, MYSQL_AUDIT_SERVER_STARTUP_CLASS = 6, MYSQL_AUDIT_SERVER_SHUTDOWN_CLASS = 7, MYSQL_AUDIT_COMMAND_CLASS = 8, MYSQL_AUDIT_QUERY_CLASS = 9, MYSQL_AUDIT_STORED_PROGRAM_CLASS = 10, These new notifications are sent from various parts of the code. I think that part can be most difficult to keep similar between Maria and My. New data structures are declared for these classes to send data inside notify(). In my opinion these data structures only contain little information, not sufficient for many tasks. 2. Plugin can specify the CLASS/SUBCLASS filter in the st_mysql_audit structure. To implement that nubers of subclasses were changed (i mean previously known subclasses). 3. notify() function returns 'int' (it used to be 'void'). I think 2. and 3. are the reasons MySQL doesn't load plugins with older interface. 4. Notification function can affect server. - Nonzero return from the notify() breaks the query execution in some cases. - Query can be modified inside the 'notify' function in some cases. 5. Sending notification looks quite crazy in the code. All notification-sending functions share the exactly same name 'mysql_audit_notify'. Naturally the number and types of parameters are different. So that i have complications if i want to find all the places where for instance the MYSQL_AUDIT_PARSE_CLASS notifications are sent. 'grep' isn't much help anymore. 6. The code is immature. The MYSQL_AUDIT_TABLE_ACCESS_CLASS just is not supported. st_mysql_audit::class_mask doesn't always work as it's supposed to. Now my proposals for our audit API. The 'radical' proposal : make the notify() function return 'int' freeze the AUDIT_INTERFACE_VERSION forever. notify() expects the 'event_class' as a parameter. It does something with the classes it is aware of. if it gets the class it doesn't know about - it should just return. notify() at the moment is expected to get the server data from the 'const void *event' sent to it as a paramter, Which is in my opinion inconvenient for everybody. My idea is to provide a set of functions like 'ap_get_query_string(MYSQL_THD thd, const void *event)', or 'ap_get_connection_id(MYSQL_THD thd, const void *event)'. That set can be extended in newer versions of the server, but old functions never change. If a plugin connects to the old server where that new possibilities are not implemented for some reason, these function still available, just do nothing. Plugin can check if some functions are available with the server it connected to. We provide the plugin developer with an 'audit_plugin_adaptor.c' file, that is built inside the plugin. It checks the version of the server that loads the plugin and links or implements the 'ap_*' functions respectively. Also it provides proper structures for the connecting server and interceipts notification's call to prepare it for the developer's notify() function. That way the plugin once built can work with variety of servers. And getting newer 'audit_plugin_adaptor.c' is going to make your plugin working with the newer versions of a server (be it MariaDB or MySQL). Less radical proposition is to mimic MySQL's API, to make the plugin compilable both in MySQL and MariaDB environment. So we get rid of the 'maria_declare_plugin', leaving only 'mysql_declare_plugin'. The present idea of having both doesn't work well. Last option i see - get rid of the 'mysql_declare_plugin', leaving only 'maria_declare_plugin' and forget about the idea that same plugin is used in MySQL and MariaDB at the same time. In this case i'd provide the set of the beforementioned 'ap_*' functions for the plugin to get server data. As before i belive it'd be convenient, we'd never need to check the plugin API version, and once built the plugin can work with many versions of MariaDB. And we don't need the 'adaptor.c' file as the proper .h file seem to be enough. Best regards. HF

1 0

[Maria-developers] convert_tz unexpectedly returning fractional seconds
by Yitzchak Scott-Thoennes 27 Jan '16

27 Jan '16

Using the ubuntu 14.04 mariadb (5.5.46), convert_tz unexpectedly returns a time with fractional seconds if its first argument is an expression using a column. This doesn't happen with mysqldb 5.5.46. Example: select convert_tz(concat('2006-05-25 17:45:00',''),'+00:00','+00:00') no_columns_used, convert_tz(concat('2006-05-25 17:45:00',blank),'+00:00','+00:00') column_used from (select '' blank) foo \G *************************** 1. row *************************** no_columns_used: 2006-05-25 17:45:00 column_used: 2006-05-25 17:45:00.000000

2 1

[Maria-developers] MDEV-9181 (NULLIF(count(table.col)), 0) gives wrong result on 10.1.x
by Alexander Barkov 26 Jan '16

26 Jan '16

Hi, Please review a new version of a patch for MDEV-9181. Thanks.

1 0

[Maria-developers] MDEV-9332 Bug after upgrade to 10.1.10
by Alexander Barkov 26 Jan '16

26 Jan '16

Hi Sergei, Please review a patch form MDEV-9332. It does the following: - Changes the behaviour of copy_if_not_alloced in case from->Alloced_length==0 and from_length==0 to return "to" instead of "from", because "from" can point to a constant string. - Rewrites the code to make it more readable (I hope), adding comments. Thanks.

2 2

Re: [Maria-developers] [Commits] e58e69e: MDEV-6421 SQL_ERROR_LOG doesn't log comments in Events.
by Sergei Golubchik 25 Jan '16

25 Jan '16

Hi, Alexey! On Jan 25, Alexey Botchkov wrote: > revision-id: e58e69e2feed36cdbffbae18f75faf0aea028401 (mariadb-10.1.10-24-ge58e69e) > parent(s): ff8d4009a7743b99c1c13831172b08041a4b272e > committer: Alexey Botchkov > timestamp: 2016-01-25 11:28:58 +0400 > message: > > MDEV-6421 SQL_ERROR_LOG doesn't log comments in Events. > Change parser so it saves all the query line to the ';' in the > sp_instr::m_query. Looks ok to push, thanks! Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org

1 0

[Maria-developers] More contributors welcome to MariaDB packaging in Debian
by Otto Kekäläinen 24 Jan '16

24 Jan '16

Hello! Do you want to improve MariaDB in Debian? Do you want to contribute to Debian packaging using modern git-buildpackage tools and state of the art QA tools? Do you want to see the result of your work in the hands of numerous of Linux users who run Debian and MariaDB? I am happy to mentor anybody who want's to start contributing to Debian packaging of MariaDB. I have created some items with priority "wishlist" and title prepended with "TODO" visible at https://bugs.debian.org/cgi-bin/pkgreport.cgi?src=mariadb-10.0#_0_6_4 These are all farily easy tasks that anybody interested in contributing could pick up and and start working on. If you want to contribute, please subscribe to the packaging teams list and show your interest there / ask questions to get directions and more help: http://lists.alioth.debian.org/pipermail/pkg-mysql-maint/ I promise to quickly review and respond to questions so that contributors feel welcomed! - Otto

3 4

[Maria-developers] bugs for beginners
by Aniket Shukla 22 Jan '16

22 Jan '16

Hello developers,I was wondering if anyone could point out some bugs for beginners in MariaDB(for the purpose of contribution) .Any help would be thankfully welcomed. Aniket

2 1

[Maria-developers] Oracle CPU January 2016
by Colin Charles 21 Jan '16

21 Jan '16

Hi! Oracle has released a Critical Patch Update: http://www.oracle.com/technetwork/topics/security/cpujan2016-2367955.html The following vulnerabilities are fixed by upgrading from MySQL 5.5.46 to 5.5.47 (picked it from the Debian maintainers list): CVE-2016-0505 CVE-2016-0546 CVE-2016-0597 CVE-2016-0598 CVE-2016-0600 CVE-2016-0606 CVE-2016-0608 CVE-2016-0609 CVE-2016-0596 CVE-2016-0616 We already have released MariaDB Server 5.5.47, so I guess its time we updated: https://mariadb.com/kb/en/mariadb/security/ (assuming that all this is fixed?) As an aside, it is interesting to see that there are CVE’s fixed for 5.7 only (see: http://www.oracle.com/technetwork/topics/security/cpujan2016-2367955.html#A…) -- Colin Charles, http://bytebot.net/blog/ twitter: @bytebot | skype: colincharles "First they ignore you, then they laugh at you, then they fight you, then you win." -- Mohandas Gandhi

1 0

Re: [Maria-developers] MDEV-8615: Assertion `m_cpp_buf <= begin_ptr && begin_ptr <= m_cpp_buf + m_buf_length' failed in Lex_input_stream::body_utf8_start
by Sergei Golubchik 21 Jan '16

21 Jan '16

Hi, Sanja! On Dec 25, OleksandrByelkin wrote: > revision-id: 684bd68c33284f6106551188d9567f35b18593ed (mariadb-10.1.9-21-g684bd68) > parent(s): 59fcd7ff2315d007045eb987da5f21abbea6f6f1 > committer: Oleksandr Byelkin > timestamp: 2015-12-25 18:39:43 +0100 > message: > > MDEV-8615: Assertion `m_cpp_buf <= begin_ptr && begin_ptr <= m_cpp_buf + m_buf_length' failed in Lex_input_stream::body_utf8_start > > Nothing should be done before any keyword recognised. > > diff --git a/mysql-test/r/mysqltest.result b/mysql-test/r/mysqltest.result > index e258b1d..a9980a7 100644 > --- a/mysql-test/r/mysqltest.result > +++ b/mysql-test/r/mysqltest.result > @@ -29,40 +29,40 @@ select 0 as "after_successful_stmt_errno" ; > after_successful_stmt_errno > 0 > garbage ; > -ERROR 42000: You have an error in your SQL syntax; <...> near 'garbage' at line 1 > +ERROR 42000: You have an error in your SQL syntax; <...> near '' at line 1 Why is that? I mean, how does the parser parse "garbage" now? > diff --git a/mysql-test/suite/funcs_1/r/storedproc.result b/mysql-test/suite/funcs_1/r/storedproc.result > index f0e1ee7..485082a 100644 > --- a/mysql-test/suite/funcs_1/r/storedproc.result > +++ b/mysql-test/suite/funcs_1/r/storedproc.result > @@ -15982,8 +15982,7 @@ insert into res_t3_itisalongname_1381742_itsaverylongname_1381742 values( 'xyz' > until count1 < 3 > END repeat label1; > END// > -ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'label1; > -END' at line 8 > +ERROR 42000: End-label label1 without match New error message is correct, great! > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index a5a62ae..74d308f 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -248,6 +248,36 @@ static bool maybe_start_compound_statement(THD *thd) > return 0; This looks quite good! But it was a bit difficult to see how the new sql_yacc.yy will look like. Did you push this patch? In what branch? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org -- Vote for my Percona Live 2016 talks: https://www.percona.com/live/data-performance-conference-2016/sessions/mari… https://www.percona.com/live/data-performance-conference-2016/sessions/mari…

2 1

[Maria-developers] MDEV-8930 main.func_hybrid_type with --embedded --ps-protocol: Assertion `(longlong) thd->status_var.local_memory_used >= 0' failed
by Alexander Barkov 20 Jan '16

20 Jan '16

Hi Sergei, IIRC you fixed this bug during the meeting in Amsterdam. Can you please confirm? (or just close it, if so) Thanks.

1 0

[Maria-developers] Request for the project
by Ayush Bhargav 20 Jan '16

20 Jan '16

Hi. I am studying programming in college and I wanted to have some hand on experience on larger projects. I know c++ and mysql well. Please can you provide me a project on which I can start working. Thanks. - Ayush Bhargav

2 1

Re: [Maria-developers] [Commits] cae1945: MDEV-9117: Client Server capability negotiation for MariaDB specific functionality
by Sergei Golubchik 20 Jan '16

20 Jan '16

Hi, Sanja! On Nov 25, OleksandrByelkin wrote: > revision-id: cae19452eec1d3be47fa04759dd2dda6d061543e (mariadb-10.1.8-69-gcae1945) > parent(s): 55e67c3e344317c6f349f5391e5d117ec51ae062 > committer: Oleksandr Byelkin > timestamp: 2015-11-25 18:09:39 +0100 > message: > > MDEV-9117: Client Server capability negotiation for MariaDB specific functionality > > New capability flags space. > Removed old progress flag, added new one. > > diff --git a/include/mysql_com.h b/include/mysql_com.h > index aa6ab0f..57092dc 100644 > --- a/include/mysql_com.h > +++ b/include/mysql_com.h > @@ -215,7 +216,8 @@ enum enum_server_command > /* Don't close the connection for a connection with expired password. */ > #define CLIENT_CAN_HANDLE_EXPIRED_PASSWORDS (1UL << 22) > > -#define CLIENT_PROGRESS (1UL << 29) /* Client support progress indicator */ > +#define CLIENT_PROGRESS_OBSOLETE (1UL << 29) ok > +#define CLIENT_PROGRESS 0 why is that? > #define CLIENT_SSL_VERIFY_SERVER_CERT (1UL << 30) > /* > It used to be that if mysql_real_connect() failed, it would delete any > @@ -228,6 +230,13 @@ enum enum_server_command > */ > #define CLIENT_REMEMBER_OPTIONS (1UL << 31) > > +/* MariaDB extended capability flags */ > +#define MARIADB_CLIENT_FLAGS_MASK 0xffffffff00000000ULL > +/* Client support progress indicator */ > +#define MARIADB_CLIENT_PROGRESS (1ULL << 32) > +/* Client support mariaDB extended flags */ > +#define MARIADB_CLIENT_EXTENDED_FLAGS (1ULL << 63) how can this work, if client_flag and server_capabilities are only 32-bit variables? > + > #ifdef HAVE_COMPRESS > #define CAN_CLIENT_COMPRESS CLIENT_COMPRESS > #else > diff --git a/sql/sql_acl.cc b/sql/sql_acl.cc > index 1240d5d..d07e0a4 100644 > --- a/sql/sql_acl.cc > +++ b/sql/sql_acl.cc > @@ -11729,18 +11731,38 @@ static ulong parse_client_handshake_packet(MPVIO_EXT *mpvio, > */ > DBUG_ASSERT(net->read_pos[pkt_len] == 0); > > - ulong client_capabilities= uint2korr(net->read_pos); > + ulonglong client_capabilities= uint2korr(net->read_pos); > + compile_time_assert(sizeof(client_capabilities) >= 8); > if (client_capabilities & CLIENT_PROTOCOL_41) > { > - if (pkt_len < 4) > + if (pkt_len < 32) > return packet_error; > client_capabilities|= ((ulong) uint2korr(net->read_pos+2)) << 16; > + if (!(client_capabilities & CLIENT_MYSQL)) > + { > + // it is client with mariadb extensions > + client_capabilities|= CLIENT_MYSQL; > + ulonglong ext_client_capabilities= > + (((ulonglong)uint4korr(net->read_pos + 28)) << 32); > + if (ext_client_capabilities & MARIADB_CLIENT_EXTENDED_FLAGS) > + client_capabilities|= ext_client_capabilities; > + else > + { > + DBUG_PRINT("error", ("CLIENT_PROTOCOL_41: on, " > + "CLIENT_LONG_PASSWORD/CLIENT_MYSQL off, " > + "but MARIADB_CLIENT_EXTENDED_FLAGS is off. " > + "flags: %llx ext flags %llx", > + client_capabilities, ext_client_capabilities)); > + return packet_error; Why do you need that (MARIADB_CLIENT_EXTENDED_FLAGS and the check)? > + } > + } here you need to check for old MariaDB clients - they will have CLIENT_MYSQL and might have CLIENT_PROGRESS_OBSOLETE - do something like else if (client_capabilities & CLIENT_PROGRESS_OBSOLETE) { client_capabilities |= MARIADB_CLIENT_PROGRESS; } > } > > diff --git a/tests/mysql_client_test.c b/tests/mysql_client_test.c > index a1a52e8..31fca0f 100644 > --- a/tests/mysql_client_test.c > +++ b/tests/mysql_client_test.c > @@ -15336,7 +15336,7 @@ static void test_mysql_insert_id() > > myheader("test_mysql_insert_id"); > > - rc= mysql_query(mysql, "drop table if exists t1"); > + rc= mysql_query(mysql, "drop table if exists t1,t2"); separate commit, please > myquery(rc); > /* table without auto_increment column */ > rc= mysql_query(mysql, "create table t1 (f1 int, f2 varchar(255), key(f1))"); > @@ -18700,7 +18700,8 @@ static void test_progress_reporting() > myheader("test_progress_reporting"); > > conn= client_connect(CLIENT_PROGRESS, MYSQL_PROTOCOL_TCP, 0); > - DIE_UNLESS(conn->client_flag & CLIENT_PROGRESS); > + if (!(conn->client_flag & CLIENT_PROGRESS)) > + return; what are you testing here? Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org -- Vote for my Percona Live 2016 talks: https://www.percona.com/live/data-performance-conference-2016/sessions/mari… https://www.percona.com/live/data-performance-conference-2016/sessions/mari…

2 8

[Maria-developers] Contribution in MariaDB
by Sanjana Garg 18 Jan '16

18 Jan '16

Hi, I am interested in contributing to the MariaDB project. Can someone guide me on how to get started. I am proficient in C, C++, python and SQL. Thanks Sanjana Garg

2 1

[Maria-developers] Projects to contribute
by Sonali Ellath 18 Jan '16

18 Jan '16

Hi Everyone, I was looking to contribute to MariaDb for GSoC 2016, I have successfully built MariaDb from source. What should be my next step? Should I pick a project and get it done? I am proficient in C++ programming, which ones would be suitable for me? Thanks! - Sonali

2 1

[Maria-developers] MDEV-8789: fully qualified field name resolving problem
by Oleksandr Byelkin 14 Jan '16

14 Jan '16

Hi! I decided to show my finding as soon as I have them found (to give time to explain/fix, also I think that name resolving fix could possibly cause some big rewriting) but if you prefer "all togather" do not hesitate to say it. So the test suite which demostrate the problem: --------- create table t1(a int, b int); insert into t1 values (1,1),(2,2),(10,10),(20,20); create table t2(c int, d int); insert into t2 values (10,1),(20,2); # mentioning "full path" to field works here select * from test.t2 where test.t2.c <20; # and does not work here with t2 as (select a,b from t1 where a < 10) select * from t2 as tt, test.t2 where tt.a = test.t2.c; drop table t1,t2; ---------- SELECT with "WITH" clause fails complaining about undefined column in the WHERE clause. BTW if remove alias tt it complain about two tables t2 when such query: select * from test2.t2, test.t2 where test2.t2.c = test.t2.c; is legal. I am not 100% sure if it is also a bug, but possible it is.

1 3

Re: [Maria-developers] For review https://mariadb.atlassian.net/browse/MDEV-9058
by Oleksandr Byelkin 13 Jan '16

13 Jan '16

On 12.01.2016 13:09, Michael Widenius wrote: >>>>>> "Oleksandr" == Oleksandr Byelkin <sanja(a)montyprogram.com> writes: > Oleksandr> Hi! > Oleksandr> Here is full diff of the task (will be tested more when georg made it in > Oleksandr> the connectors tree) > > Oleksandr> Also it can be reviewed by parts in bb-sanja-10.2 branch on github. > > Oleksandr> Connectors part is on 10.2-georg in connectors tree. > > > diff --git a/include/mysql_com.h b/include/mysql_com.h > index 57092dc..425559f 100644 > --- a/include/mysql_com.h > +++ b/include/mysql_com.h > @@ -102,7 +102,9 @@ enum enum_server_command > COM_STMT_PREPARE, COM_STMT_EXECUTE, COM_STMT_SEND_LONG_DATA, COM_STMT_CLOSE, > COM_STMT_RESET, COM_SET_OPTION, COM_STMT_FETCH, COM_DAEMON, > /* don't forget to update const char *command_name[] in sql_parse.cc */ > - > + COM_MDB_GAP_BEG, > + COM_MDB_GAP_END=254, > + COM_MULTI, > /* Must be last */ > COM_END > }; > > Change COM_MDB_GAP_END to be 253, to ensure that COM_END is 255. > (Better to keep everything in one byte) > > Also add an compiler error check that COM_END is 255 (to ensure that one > doesn't accidently add a new define without changing COM_DB_GAP_END. OK! > > @@ -234,6 +236,8 @@ enum enum_server_command > #define MARIADB_CLIENT_FLAGS_MASK 0xffffffff00000000ULL > > Haven't seen the above define before and it's not in the current 10.1 > code. ah, it was removed wit client code, so I'll remove it if it is not used. > > /* Client support progress indicator */ > #define MARIADB_CLIENT_PROGRESS (1ULL << 32) > +/* support COM_MULTI */ > +#define MARIADB_CLIENT_COM_MULTI (1ULL << 33) > /* Client support mariaDB extended flags */ > #define MARIADB_CLIENT_EXTENDED_FLAGS (1ULL << 63) > > @@ -268,7 +272,8 @@ enum enum_server_command > MARIADB_CLIENT_PROGRESS | \ > CLIENT_PLUGIN_AUTH | \ > CLIENT_PLUGIN_AUTH_LENENC_CLIENT_DATA | \ > - CLIENT_CONNECT_ATTRS) > + CLIENT_CONNECT_ATTRS |\ > + MARIADB_CLIENT_COM_MULTI) > > You don't have MARIADB_CLIENT_COM_MULTI anywhere else in the patch. > Is that intententional ? > > If yes, you should add a comment that this is used in the LGPL C > driver code so that anyone examening the server code will not get > confused. > > As you have added flags bigger than 32, you should also change > THD::client_capabilities from ulong to ulonglong to reflect this! > > This patch doesn't include changes in sql_acl.cc to send the > MARIADB_CLIENT_COM_MULTI flag to the client. > Where are the necessary changes for this ? expanding in to 64bit was separate task (reviewed and pushed). Also we are not going to support this new features in old client, so we are signaling our connector only. I'll write a comment about it. > > diff --git a/sql/sql_class.cc b/sql/sql_class.cc > index b496e4c..54884c0 100644 > --- a/sql/sql_class.cc > +++ b/sql/sql_class.cc > @@ -856,6 +856,7 @@ THD::THD(bool is_wsrep_applier) > /* statement id */ 0), > rli_fake(0), rgi_fake(0), rgi_slave(NULL), > protocol_text(this), protocol_binary(this), > + last_stmt(NULL), > in_sub_stmt(0), log_all_errors(0), > binlog_unsafe_warning_flags(0), > binlog_table_maps(0), > > You don't have to set last_stmt(NULL) in THD::THD() as this function > calls init() which sets this too. ok > > @@ -1424,6 +1425,7 @@ void THD::init(void) > bzero((char *) &org_status_var, sizeof(org_status_var)); > start_bytes_received= 0; > last_commit_gtid.seq_no= 0; > + last_stmt= NULL; > status_in_global= 0; > #ifdef WITH_WSREP > wsrep_exec_mode= wsrep_applier ? REPL_RECV : LOCAL_STATE; > > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > @@ -884,6 +1115,23 @@ void cleanup_items(Item *item) > DBUG_VOID_RETURN; > } > > +static enum enum_server_command fetch_command(THD *thd, char *packet) > +{ > + enum enum_server_command > + command= (enum enum_server_command) (uchar) packet[0]; > + NET *net= &thd->net; > + DBUG_ENTER("fetch_command"); > + > > If command is 256, then the following first test can never be true and > you may get a warning from gcc. Another reason to have COM_END as 255. yes > > + if (command >= COM_END || > + (command >= COM_MDB_GAP_BEG && command <= COM_MDB_GAP_END)) > + command= COM_END; // Wrong command > + > + DBUG_PRINT("info",("Command on %s = %d (%s)", > + vio_description(net->vio), command, > + command_name[command].str)); > + DBUG_RETURN(command); > +} > + > > /** > + check COM_MULTI packet > + > + @param thd thread handle > + @param packet pointer on the packet of commands > + @param packet_length length of this packet > + > + @retval TRUE - Error > + @retval FALSE - OK > +*/ > + > +bool maria_multi_check(THD *thd, char *packet, uint packet_length) > +{ > + DBUG_ENTER("maria_multi_check"); > + while (packet_length) > + { > > You need to check for packet_length >= 3 and give the > my_message(ER_UNKNOWN_COM_ERROR... error if packet_length > 1 and <= 3 > > > + // length of command + 3 bytes where that length was stored > + uint subpacket_length= (uint3korr(packet) + 3); > + DBUG_PRINT("info", ("sub-packet length: %d command: %x", > + subpacket_length, packet[3])); > + > + if (subpacket_length == 0 || > + subpacket_length > packet_length) > > Remove check of sub_packet_length == 0; It can never be 0 as you are > adding +3 above. ok > > + { > + my_message(ER_UNKNOWN_COM_ERROR, ER_THD(thd, ER_UNKNOWN_COM_ERROR), > + MYF(0)); > + DBUG_RETURN(TRUE); > + } > + > + packet+= subpacket_length; > + packet_length-= subpacket_length; > + } > > + DBUG_RETURN(FALSE); > +} > > > @@ -1901,6 +2200,69 @@ bool dispatch_command(enum enum_server_command command, THD *thd, > general_log_print(thd, command, NullS); > my_eof(thd); > break; > + case COM_MULTI: > + if (is_com_multi) > + { > + /* we do not allow deep recursion because it is not safe */ > + my_message(ER_UNKNOWN_COM_ERROR, ER_THD(thd, ER_UNKNOWN_COM_ERROR), > + MYF(0)); > + break; > + } > > Give another error as the above one is missleading. OK > > + if (!(thd->client_capabilities & CLIENT_MULTI_RESULTS)) > + { > + /* The client does not support multiple result sets being sent back */ > + my_error(ER_COMMULTI_BADCONTEXT, MYF(0)); > + break; > + } > > How can the above be possible ? > We should not allow the client to set COM_MULTI flag if they don't > have CLIENT_MULTI_RESULT. > Better to remove the above check and document when the client can set the > COM_MULTI flag. I am not of control of client, it can be some home-made client so better to check. > > + if (maria_multi_check(thd, packet, packet_length)) > + break; > + { > + /* We have to store next length because it will be destroyed by '\0' */ > + uint next_subpacket_length= uint3korr(packet); > + unsigned char *readbuff= net->buff; > + unsigned long readbuff_max_packet= net->max_packet; > + > > Remove the above one extra empty line. > Add also some documentation why we are changing net->buff here instead > of creating another buffer and using this instead. > > Actually it's much better to create a new buffer for packet becasue > of: > > - The buffer only need to be of size packet_length, not > net->max_packet which can be much longer. > - Clearer code as you don't have to manipulate net internal structures > or restore these. > - The disadvantage is that you can't allow COM_CHANGE_USER, but I > don't think one should be able to do that anyway as this may ask for > more information from the client. OK > > > + if (!(net->buff=(uchar*) my_malloc((size_t) net->max_packet+ > + NET_HEADER_SIZE + COMP_HEADER_SIZE +1, > + MYF(MY_WME)))) > + break; > + net->buff_end=net->buff + net->max_packet; > + net->write_pos=net->read_pos = net->buff; > + > + while(packet_length) > > space after while > > + { > + uint subpacket_length= next_subpacket_length; > > Why not do as in other code and make the rest of the code easier: > subpacket_length= next_subpacket_length +3 ; ok > > + if (subpacket_length + 3 < packet_length) > + next_subpacket_length= uint3korr(packet + subpacket_length + 3); > + /* safety like in do_command() */ > + packet[subpacket_length + 3]= '\0'; > + > + enum enum_server_command subcommand= fetch_command(thd, (packet + 3)); > + > > I would almost prefer that we check for sub_commend == COM_MULTI here > instead of at start of COM_COMMAND: This would be more logical as we > may not allow all commands here (like change user). OK > > + if (is_com_multi) > + { > + /* we do not allow deep recursion because it is not safe */ > + my_message(ER_UNKNOWN_COM_ERROR, ER_THD(thd, ER_UNKNOWN_COM_ERROR), > + MYF(0)); > + break; > + } > > > + if (dispatch_command(subcommand, thd, packet + 4, > + subpacket_length - 1, TRUE)) > > Add an assert that is_error() is set if we do a break. > (As otherwise you can miss to execute commands the client expects) OK > > + break; > + > + DBUG_ASSERT(subpacket_length + 3 <= packet_length); > + packet+= (subpacket_length + 3); > + packet_length-= (subpacket_length + 3); > > + } > > I don't think you need a net_flush here as the packet is flushed by > every command the sent a result. OK > > + net_flush(net); > + my_free(net->buff); > + net->buff= readbuff; > + net->max_packet= readbuff_max_packet; > + net->buff_end=net->buff + net->max_packet; > + net->write_pos=net->read_pos = net->buff; > + > > If we use an internal buffer, you can remoev all of the above (except > the free) OK > > + if (!thd->is_error()) > + { > + thd->get_stmt_da()->reset_diagnostics_area(); > + my_ok(thd); > + } > + } > + break; > > case COM_SLEEP: > case COM_CONNECT: // Impossible here > case COM_TIME: // Impossible from client > @@ -1940,6 +2302,8 @@ bool dispatch_command(enum enum_server_command command, THD *thd, > thd->protocol->end_statement(); > query_cache_end_of_result(thd); > } > + if (drop_more_results) > + thd->server_status&= ~SERVER_MORE_RESULTS_EXISTS; > > Another way, is to just remember the status of server_status and do: > > thd->server_status= ((thd->server_status & ~SERVER_MORE_RESULTS_EXISTS) | > (org_server_status & SERVER_MORE_RESULTS_EXISTS)); > > Easier to understand than drop_more_results I am not sure that execution can't legally change setver status (if not now then in future) > > --- a/sql/sql_prepare.cc > +++ b/sql/sql_prepare.cc > @@ -324,7 +324,9 @@ find_prepared_statement(THD *thd, ulong id) > prepared statements find() will return 0 if there is a named prepared > statement with such id. > */ > - Statement *stmt= thd->stmt_map.find(id); > + Statement *stmt= ((id == LAST_STMT_ID) ? > + thd->last_stmt : > + thd->stmt_map.find(id)); > > Please add a comment when id can be LAST_STMT_ID (clear to me but not > obivous for others) > > > if (stmt == 0 || stmt->type() != Query_arena::PREPARED_STATEMENT) > return NULL; > > --- a/storage/perfschema/pfs.cc > +++ b/storage/perfschema/pfs.cc > @@ -1444,19 +1444,21 @@ static void register_statement_v1(const char *category, > > for (; count>0; count--, info++) > { > - DBUG_ASSERT(info->m_name != NULL); > - len= strlen(info->m_name); > - full_length= prefix_length + len; > - if (likely(full_length <= PFS_MAX_INFO_NAME_LENGTH)) > + if(info->m_name != NULL) > > Add space after if > > Instead of doing an if, and cause a big diff, do instead: > > if (!info->m_name) > continue; > > This will remove all of the following in the diff: OK > > > { > - memcpy(formatted_name + prefix_length, info->m_name, len); > - info->m_key= register_statement_class(formatted_name, full_length, info->m_flags); > - } > - else > - { > - pfs_print_error("register_statement_v1: name too long <%s>\n", > - info->m_name); > - info->m_key= 0; > + len= strlen(info->m_name); > + full_length= prefix_length + len; > + if (likely(full_length <= PFS_MAX_INFO_NAME_LENGTH)) > + { > + memcpy(formatted_name + prefix_length, info->m_name, len); > + info->m_key= register_statement_class(formatted_name, full_length, info->m_flags); > + } > + else > + { > + pfs_print_error("register_statement_v1: name too long <%s>\n", > + info->m_name); > + info->m_key= 0; > + } > } > } > return; > > Regards, > Monty

1 1