- developers - lists.mariadb.org

[Maria-developers] MDEV-8433 Make field<'broken-string' use indexes
by Alexander Barkov 07 Jul '15

07 Jul '15

Hi Sergei, I created a task according to our discussion a few weeks ago: https://mariadb.atlassian.net/browse/MDEV-8433 Which version should we do this task in? Thanks.

1 0

Re: [Maria-developers] GSoc2015: MDEV-8356 first patch
by Sergey Petrunia 03 Jul '15

03 Jul '15

Hi Diwas, On Fri, Jul 03, 2015 at 02:54:28PM +0530, Diwas Joshi wrote: > hello sergey, I have fixed the issues from the previous patch in this one. > Please find some feedback below. After you have addressed it, please commit the change and push it to your tree. The next task to focus on after this one is MDEV-8389. > diff --git a/mysql-test/r/sp-error.result b/mysql-test/r/sp-error.result > index 897895e..cacfeea 100644 > --- a/mysql-test/r/sp-error.result > +++ b/mysql-test/r/sp-error.result > @@ -2866,12 +2866,6 @@ SELECT @msg; > DROP FUNCTION f1; > DROP FUNCTION f2; > DROP TABLE t1; > -CREATE FUNCTION f1(a INT, b VARCHAR(11)) > -RETURNS TABLE t1(id INT, name VARCHAR(11)) > -BEGIN > -INSERT INTO t1 SELECT id, name FROM t2 WHERE id = a; > -END| > -ERROR 42000: This version of MariaDB doesn't yet support 'Table functions' > SELECT * FROM f1('xyz'); > ERROR 42000: This version of MariaDB doesn't yet support 'Queries using Table Functions' > SELECT * FROM t1, f1('xyz',t1.a); > diff --git a/mysql-test/r/sp.result b/mysql-test/r/sp.result > index 3f28df9..2a931b1 100644 > --- a/mysql-test/r/sp.result > +++ b/mysql-test/r/sp.result > @@ -7937,3 +7937,17 @@ return 1; > end | > ERROR 0A000: Not allowed to return a result set from a function > drop table t1,t2; > +# Tests for Table functions > +CREATE FUNCTION f1(a INT, b VARCHAR(11)) > +RETURNS TABLE t1(id INT, name VARCHAR(11)) > +BEGIN > +INSERT INTO t1 SELECT id, name FROM t2 WHERE id = a; > +END| Please add "--vertical_results" here so that the SELECT results are printed vertically. It will be easier to see what the return value is. > +SELECT * FROM mysql.proc WHERE db = database() and name = 'f1'; > +db name type specific_name language sql_data_access is_deterministic security_type param_list returns body definer created modified sql_mode comment character_set_client collation_connection db_collation body_utf8 > +test f1 FUNCTION f1 SQL CONTAINS_SQL NO DEFINER a INT, b VARCHAR(11) TABLE t1(id int(11), name varchar(11)) BEGIN > +INSERT INTO t1 SELECT id, name FROM t2 WHERE id = a; > +END root@localhost 2015-07-03 12:18:04 2015-07-03 12:18:04 latin1 latin1_swedish_ci latin1_swedish_ci BEGIN > +INSERT INTO t1 SELECT id, name FROM t2 WHERE id = a; > +END > +DROP FUNCTION f1; -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] Buildbot branch 10.0-custombld
by Kristian Nielsen 02 Jul '15

02 Jul '15

Hi Rasmus, The branch 10.0-custombld seems to no longer build binaries (after the transition to git?): https://github.com/MariaDB/server/tree/10.0-custombld https://buildbot.askmonty.org/buildbot/grid?branch=10.0-custombld&category=… Can you please fix Buildbot so that this branch works again for producing custom builds of all packages? This is urgent, as it is blocking further progress of MDEV-8302. Thanks, - Kristian.

1 1

Re: [Maria-developers] GSoc2015: MDEV-8356 first patch
by Sergey Petrunia 02 Jul '15

02 Jul '15

Hi Diwas, On Thu, Jul 02, 2015 at 04:26:58PM +0530, Diwas Joshi wrote: > for the return type i am storing a string containing TABLE(<col > definitions>). eg. forCREATE FUNCTION f1(a INT, b VARCHAR(11)) > RETURNS TABLE t1(id INT, name VARCHAR(11)) BEGIN set @a=3; END| > we store > TABLE t1(id INT, name VARCHAR(11)) Ok. > for the type of function we can store it as TYPE_ENUM_FUNCTION only and > later see if it creates a problem, although we can always differentiate > between the functions based on the return type stored above. Yes, let's keep it this way for now. > > I included the table alias with the return type string as you suggested. > There is a warning that i am getting, which i am failing to figure the > reason for but I am still trying. > Please review. > The first thing: please add a testcase that checks what the code code does. I can think of doing this: CREATE table_function .... < whatever here> select * from mysql.proc where name='table_function'; This will show what was stored in the system table. This will also make it clear if/what warning gets produced. Since you've changed CREATE FUNCTION table_function ... not to return error, I guess some of testcases you previously added will need to be modified, too. Overall the patch seems to be ok but I would like to look at the final version of it before giving the approval. > diff --git a/sql/sp.cc b/sql/sp.cc > index 9fd17b0..3413ba1 100644 > --- a/sql/sp.cc > +++ b/sql/sp.cc > @@ -925,21 +925,66 @@ class Bad_db_error_handler : public Internal_error_handler > bzero((char*) &share, sizeof(share)); > table.in_use= thd; > table.s = &share; > - field= sp->create_result_field(0, 0, &table); > - field->sql_type(result); > - > - if (field->has_charset()) > + if (sp->m_type == TYPE_ENUM_FUNCTION) > { > - result.append(STRING_WITH_LEN(" CHARSET ")); > - result.append(field->charset()->csname); > - if (!(field->charset()->state & MY_CS_PRIMARY)) > + field= sp->create_result_field(0, 0, &table); > + field->sql_type(result); > + > + if (field->has_charset()) > { > - result.append(STRING_WITH_LEN(" COLLATE ")); > - result.append(field->charset()->name); > + result.append(STRING_WITH_LEN(" CHARSET ")); > + result.append(field->charset()->csname); > + if (!(field->charset()->state & MY_CS_PRIMARY)) > + { > + result.append(STRING_WITH_LEN(" COLLATE ")); > + result.append(field->charset()->name); > + } > } > + delete field; > + } > + else if (sp->m_type == TYPE_ENUM_TABLE) > + { > + List_iterator_fast<Create_field> it(sp->m_cols_list); > + Create_field *cols_field; > + cols_field= it++; > + result.append(STRING_WITH_LEN("TABLE ")); > + result.append(sp->m_table_alias.str); > + result.append(STRING_WITH_LEN("(")); > + while (cols_field) > + { > + uint unused1= 0; > + sp_prepare_create_field(thd, cols_field); > + prepare_create_field(cols_field, &unused1, HA_CAN_GEOMETRY); > + uint field_length; > + Field *field; > + String tmp_result(64); > + field_length= cols_field->length; > + cols_field->flags= 0; > + field= make_field(table.s, /* TABLE_SHARE ptr */ > + (uchar*) 0, /* field ptr */ > + field_length, /* field [max] length */ > + (uchar*) "", /* null ptr */ > + 0, /* null bit */ > + cols_field->pack_flag, > + cols_field->sql_type, > + cols_field->charset, > + cols_field->geom_type, cols_field->srid, > + Field::NONE, /* unreg check */ > + cols_field->interval, > + cols_field->field_name); > + field->vcol_info= cols_field->vcol_info; > + field->stored_in_db= cols_field->stored_in_db; > + if (field) > + field->init(&table); > + field->sql_type(tmp_result); > + result.append(tmp_result); > + cols_field= it++; > + if(cols_field) > + result.append(STRING_WITH_LEN(", ")); > + delete field; > + } > + result.append(STRING_WITH_LEN(")")); > } > - > - delete field; > } > > > @@ -1020,7 +1065,8 @@ class Bad_db_error_handler : public Internal_error_handler > char definer_buf[USER_HOST_BUFF_SIZE]; > LEX_STRING definer; > ulonglong saved_mode= thd->variables.sql_mode; > - MDL_key::enum_mdl_namespace mdl_type= type == TYPE_ENUM_FUNCTION ? > + MDL_key::enum_mdl_namespace mdl_type= type == (type == TYPE_ENUM_FUNCTION || > + type == TYPE_ENUM_TABLE) ? > MDL_key::FUNCTION : MDL_key::PROCEDURE; Looks reasonable. > > CHARSET_INFO *db_cs= get_default_db_collation(thd, sp->m_db.str); > @@ -1036,7 +1082,8 @@ class Bad_db_error_handler : public Internal_error_handler > retstr.set_charset(system_charset_info); > > DBUG_ASSERT(type == TYPE_ENUM_PROCEDURE || > - type == TYPE_ENUM_FUNCTION); > + type == TYPE_ENUM_FUNCTION || > + type == TYPE_ENUM_TABLE); > > /* Grab an exclusive MDL lock. */ > if (lock_object_name(thd, mdl_type, sp->m_db.str, sp->m_name.str)) > @@ -1117,7 +1164,8 @@ class Bad_db_error_handler : public Internal_error_handler > > store_failed= store_failed || > table->field[MYSQL_PROC_MYSQL_TYPE]-> > - store((longlong)type, TRUE); > + store((longlong)(type == TYPE_ENUM_TABLE ? > + TYPE_ENUM_FUNCTION : type), TRUE); > > store_failed= store_failed || > table->field[MYSQL_PROC_FIELD_SPECIFIC_NAME]-> > @@ -1145,7 +1193,7 @@ class Bad_db_error_handler : public Internal_error_handler > table->field[MYSQL_PROC_FIELD_PARAM_LIST]-> > store(sp->m_params.str, sp->m_params.length, system_charset_info); > > - if (sp->m_type == TYPE_ENUM_FUNCTION) > + if (sp->m_type == TYPE_ENUM_FUNCTION || sp->m_type == TYPE_ENUM_TABLE) > { > sp_returns_type(thd, retstr, sp); > > @@ -1153,7 +1201,7 @@ class Bad_db_error_handler : public Internal_error_handler > table->field[MYSQL_PROC_FIELD_RETURNS]-> > store(retstr.ptr(), retstr.length(), system_charset_info); > } > - > + String body_with_alias(64); What is this for? It's not used anywhere. > store_failed= store_failed || > table->field[MYSQL_PROC_FIELD_BODY]-> > store(sp->m_body.str, sp->m_body.length, system_charset_info); > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index b53d939..d7d7d7b 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -16392,12 +16392,6 @@ sf_tail: > Lex_input_stream *lip= YYLIP; > > lex->sphead->set_body_start(thd, lip->get_cpp_tok_start()); > - > - if (Lex->sphead->m_type== TYPE_ENUM_TABLE) //to be removed later. > - { > - my_error(ER_NOT_SUPPORTED_YET, MYF(0), "Table functions"); > - MYSQL_YYABORT; > - } > } > sp_proc_stmt /* $15 */ > { BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] GSoC2015: MDEV-8356 patch for review
by Diwas Joshi 02 Jul '15

02 Jul '15

Hello all, I am attaching a patch for MDEV-8356 <https://mariadb.atlassian.net/browse/MDEV-8356> as part of my work under GSoC2015 implementing table valued functions. Please feel free to point out mistakes, if any. regards Diwas Joshi

1 0

[Maria-developers] [GSoC] MDEV-4674 Port InnoDB memcached interface to MariaDB - Midterm report
by Piotr Jurkiewicz 02 Jul '15

02 Jul '15

Hi all, as the midterm is approaching, I am posting a report on what I have done so far. The project is going as scheduled. You can see the timeline in my proposal: http://www.google-melange.com/gsoc/proposal/public/google/gsoc2015/piotrjur… All goals planned before the midterm have been reached. InnoDB Memcached plugin ported from MySQL works now with MariaDB (both with InnoDB and XtraDB engines). That means that MariaDB is now on par with MySQL on this issue. My repository with the code is available here: https://github.com/piotrjurkiewicz/mariadb-server What have been done in details: 1. Code of InnoDB Memcached plugin has been imported from MySQL into the MariaDB tree. 2. There were some differences between MariaDB and MySQL, which blocked the plugin from out-of-the-box compilation and correct operation. They have been resolved. 3. Test suite for the plugin has been imported from MySQL. Compatibility changes were introduced into these test cases which required them. The test suite was modified to assign memcached port number dynamically from mtr worker's pool, what allows to run the same test in parallel for many combinations. The plugin fully passes the test suite, both for 'innodb_plugin' and 'xtradb' combinations. 4. Some enhancements and bug fixes have been introduced into the plugin: - detection of memcached daemon initialization has been fixed -- this eliminated unnecessary wait (15 seconds) on plugin shutdown and memory leak (in cases when plugin failed to initialize properly before) - memcached arithmetic commands (incr/decr) handling has been improved -- now numeric columns values are handled in a consistent and predictable way - incorrect handling of unsigned not null integers columns has been fixed For details of all changes please see commit messages associated with them: https://github.com/piotrjurkiewicz/mariadb-server/commits/10.1-memcached After the midterm, I plan to focus on further enhancements of the plugin, as stated in my proposal: 1. Change plugin architecture to "one table per socket", as originally proposed in MDEV-4674 bug report. This will allow usage of well known security solutions (firewalls, unix file permissions) to restrict access to different tables for different users. 2. Implement a new interface inside the Memcached plugin, which will use Handler API for database access. This will make possible to use the plugin with database engines other than InnoDB/XtraDB. Ultimately, this interface will replace existing InnoDB direct API interface (if there will be no significant performance differences). Therefore, I would like to as you to point me to Handler API documentation (if documentation such exists). I know that HandlerSocket plugin uses the Handler API to access databases. Is HandlerSocket plugin a good example of Handler API usage to follow? Piotr

1 0

[Maria-developers] How to read of innodb_locks table by using low level interface?
by 김경남 01 Jul '15

01 Jul '15

Hi, Sergei I'm developing a mariadb monitoring plugin type agent. We're planning to contribute our monitoring solution for mariadb in one or two months. Our solution is consist of plugin type agent, APM Server and Dashboard. Our plugin type agent collect data by using low level interface instead of using query. I can collect a information_schema.global_status and performance_schema.event_xxxx_table for your help in the past. By referencing the source of feedback.cc, I can collect data from global_status using fill_stauts function. I hava a question. I need to collect data from information_schema.innodb_locks for lock monitoring. But I found that I must use trx_i_s_common_fill_table in i_s.cc for reading of innodb_locks table and that fuction is static. so I only access that function by function pointer. Debugging source for many days, I guess that I must use ST_SCHEMA TABLE's fill_table member variable for pointing to trx_i_s_common_fille_table function. Right? But I don't know how to that. I attemped in various ways for many days. but I failed. If possible. please let me know how to read data of information_schema.innodb_locks using native api. Thanks in advance. Regards, Sergei. _____ 상기 메시지와 첨부화일 내에는 비밀정보가 포함되어 있을 수 있으며, 지정된 수신자에 한하여 조회 및 사용될 수 있습니다. 만약 송신자의 실수로 인하여 상기 메시지를 수신하였다면, 송신자에게 메시지를 반송해 주시고, 원본 메시지와 모든 사본을 폐기해 주시기 바랍니다. 상기 메시지의 전체 또는 일부에 대해 무단 열람, 사용, 공개, 배포하는 것은 금지되어 있습니다.(주)LG CNS. This message and its attachments may contain confidential information, and they are intended to be viewed or used by only the individuals specified in the message. If you have received this message in an error from the sender, please contact the sender immediately to notify the error and delete all of the message and its copies. It is prohibited to view, use, make public and/or distribute part or whole of this message without written permission.

1 0

[Maria-developers] sprint tasks for the next week
by Alexander Barkov 01 Jul '15

01 Jul '15

Hi Sergei, I'd like to do these tasks: - MDEV-8215 Asian MB3 charsets: compare broken bytes as "greater than any non-broken character" - MDEV-8036 Fix all collations to compare broken bytes as "greater than any non-broken character" to close the group of tasks related to handling bad bytes in comparison. Any objections? How do I mark tasks as "sprint"? Thanks.

1 0

Re: [Maria-developers] MDEV-8356 analysis and ideas
by Sergey Petrunia 29 Jun '15

29 Jun '15

Hi Diwas, Nice analysis. Please find my comments below. On Mon, Jun 29, 2015 at 10:01:56AM +0530, Diwas Joshi wrote: > hello sergey, so i was reading further on MDEV-8356 last night. There are > two main things here: type of sf and return type of sf. Right now the > create function... is giving errors at > > store_failed= store_failed || > table->field[MYSQL_PROC_MYSQL_TYPE]-> > store((longlong)type, TRUE); > > in sf_create_routine(), which is right since we have been been using > TYPE_ENUM_TABLE for m_type so far and mysql.proc takes > enum('FUNCTION','PROCEDURE') for type. This can be solved by either > changing this enum in mysql.proc definition and including TABLE, or storing > table_valued functions as FUNCTION as well. I think first option is better, > including TABLE in the enum, since this will help us in distinguishing > these functions while executing later if needed. It is not clear for me which option is better. On one hand, there is your argument that adding 'TABLE' into the enum will make it easier to tell table functions from regular function. On the other hand, try grepping for 'TYPE_ENUM_FUNCTION' in sql/*.{h,cc}. You find a lot of code that checks if m_type == TYPE_ENUM_FUNCTION. You will need to review each check to see what should be done for TYPE_ENUM_FUNCTION. If we can't decide towards one option or another, let's use whatever you prefer right now. If we discover that that works poorly, we will be able to reverse the decision later on. > The second thing is about return type of function. Right now > sp_returns_type() method is being used to store the return type of > functions. But in our case we have a list of create_fields: > List<Create_field> m_cols_list keeping list of attribute names and their > types for the table. We need to store all these, for this we can either > alter mysql.proc definition to make a field appropriate for storing this > list. The type of the field storing return type right now is longblob which > as i read further on it is a binary large object that can hold a variable > amount of data and are treated as binary strings (byte strings). Maybe we > can try to find a way to store all the attribute information in this field > only, maybe putting in some delimiters between different field types or > some other way since after all its a string. I am still not sure about this > problem. I tried creating this function: CREATE FUNCTION f2(a INT, b VARCHAR(11)) RETURNS VARCHAR(32) DETERMINISTIC BEGIN set @a=3; RETURN @a; END| and mysql.proc.returns has: returns: varchar(32) CHARSET latin1 So, it is a text defition of the return type. It seems natural if for table functions we Is it a problem to do similar thing for Table functions? e.g for a function CREATE FUNCTION f1(a INT, b VARCHAR(11)) RETURNS TABLE t1(id INT, name VARCHAR(11)) BEGIN set @a=3; END| store 'TABLE t1(id INT, name VARCHAR(11)' ? I guess it depends on where the value of mysql.proc.returns is used. Please research this. > > Also, another thing is to store the table alias m_table_alias, because > queries inside the function will be using it. There is no field present > specifically for this in mysql.proc. So either we can create one. There is > a field specific_name in it, as i saw in the code is right now storing the > same thing as name field. So maybe we can use this field to store the table > alias. I would prefer that m_table_alias was stored as part of the expression in mysql.proc.returns field. Can we try that option first? If we figure that can't work, we can think of other options. BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] MariaDB multi-source replication testing at Booking.com
by Karoly Nagy 29 Jun '15

29 Jun '15

Dear MariaDB developers, I'm Károly Nagy working for Booking.com currently testing multi-source replication functionality of MariaDB. Kristian suggested I should reach out to you on this mailing list regarding my questions. We're seeing very high and fluctuating mutex contentions while replicating from two sources (Oracle MySQL 5.6) to a single MariaDB slave. You can see that on the graphs below. The spin waits are relatively [1] aligned but the mutex rounds [2] are 5-10 times higher than it is on the two sources combined together and not consistent. The sources have a relatively constant pattern while the target has dips around 2.5k and spikes up to 8k. The os waits are in completely different order of magnitude [3]. The scenario where values were captured: * Multi-source target is replicating the full dataset of `source 2` and a subset of `source 1` (the hot data) - MariaDB 10.0.16 * Both sources are MySQL 5.6 being part of their replication chain as slaves with log_slave_updates * Source 2 is in normal mode - Oracle MySQL 5.6.17 * Source 1 is catching up from a 1 day replication delay - Oracle MySQL 5.6.24 * All the slaves are warm having the buffer pool fully populated Is this behavior expected? Could you give us some insights on why we could see these results? If there is any more information you need please let me know. Thank you for your help in advance! Every metric is on /10 seconds basis. [1] Mutex spin waits [2] Mutex rounds [3] Mutex OS waits Best regards, -- Károly Nagy System engineer Booking.com <http://booking.com/> BV Rembrandt Square Office, Herengracht 597, 1017 CE Amsterdam Direct +31 (0)20 715 8403

3 4