March 2015 - developers - lists.mariadb.org

Re: [Maria-developers] [Commits] Rev 4382: MDEV-6892: WHERE does not apply in file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6892/
by Sergey Petrunia 11 Mar '15

11 Mar '15

Hi Sanja, Please find my feedback below. We should discuss it. On Fri, Dec 19, 2014 at 01:24:12PM +0000, sanja(a)askmonty.org wrote: > At file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6892/ > > ------------------------------------------------------------ > revno: 4382 > revision-id: sanja(a)askmonty.org-20141219132350-6bo1g7q6gbivqdky > parent: svoj(a)mariadb.org-20141201105829-ctaoe194abtorhz2 > committer: sanja(a)askmonty.org > branch nick: work-maria-5.5-MDEV-6892 > timestamp: Fri 2014-12-19 14:23:50 +0100 > message: > MDEV-6892: WHERE does not apply > > Taking into account implicit dependence of constant view field from nullable table of left join added. > > Fixed finding real table to check if ut turned to NULL (materialized view & derived taken into account) ... > === modified file 'sql/item.cc' > --- a/sql/item.cc 2014-10-06 17:53:55 +0000 > +++ b/sql/item.cc 2014-12-19 13:23:50 +0000 > @@ -9790,12 +9790,36 @@ void Item_ref::update_used_tables() > (*ref)->update_used_tables(); > } > > +void Item_direct_view_ref::update_used_tables() > +{ > + if (view->is_inner_table_of_outer_join()) > + { > + null_ref_table= NULL; > + check_null_ref(); update_used_tables() is called during optimization. Why does it call check_null_ref() which may check null_ref_table->null_row? Is check_null_ref() function an execution-time function (and so it computes "current" value of an item) or is it an optimization-time function (and so it computes static attributes and can be called at any point in the query) It looks like it's a bit of both, which is really confusing. Could you please clarify? check_null_ref() was introduced by you in another views+outer joins bugfix. Can we get it documented? > + } > + Item_direct_ref::update_used_tables(); > +} Overall, I don't understand why some checks are in used_tables() and some are in update_used_tables(). Could you clarify? > + > table_map Item_direct_view_ref::used_tables() const > { > + TABLE *null_ref= null_ref_table; > + > + if (view->is_inner_table_of_outer_join()) > + { > + if (null_ref == NULL) > + null_ref= view->get_real_join_table(); > + else if (null_ref == NO_NULL_TABLE) > + null_ref= NULL; > + } > + else > + null_ref= NULL; > + > return get_depended_from() ? > OUTER_REF_TABLE_BIT : > ((view->is_merged_derived() || view->merged || !view->table) ? > - (*ref)->used_tables() : > + ((*ref)->used_tables() ? > + (*ref)->used_tables() : > + (null_ref ? null_ref->map : (table_map)0 )) : > view->table->map); Two-level nesting of conditional expressions is too complicated. Please change into 'if' statement and a comment about what's going on. How can Item_direct_view_ref() deped on an outer table?? > } > > > === modified file 'sql/item.h' > --- a/sql/item.h 2014-11-18 14:42:40 +0000 > +++ b/sql/item.h 2014-12-19 13:23:50 +0000 > @@ -3319,7 +3319,8 @@ class Item_direct_view_ref :public Item_ > { > if (null_ref_table == NULL) > { > - if (!(null_ref_table= view->get_real_join_table())) > + if (!view->is_inner_table_of_outer_join() || > + !(null_ref_table= view->get_real_join_table())) > null_ref_table= NO_NULL_TABLE; > } > if (null_ref_table != NO_NULL_TABLE && null_ref_table->null_row) > @@ -3329,6 +3330,7 @@ class Item_direct_view_ref :public Item_ > } > return FALSE; > } > + > public: > Item_direct_view_ref(Name_resolution_context *context_arg, Item **item, > const char *table_name_arg, > @@ -3353,8 +3355,10 @@ public: > bool subst_argument_checker(uchar **arg); > Item *equal_fields_propagator(uchar *arg); > Item *replace_equal_field(uchar *arg); > - table_map used_tables() const; > + table_map used_tables() const; > + void update_used_tables(); > table_map not_null_tables() const; > + bool const_item() const { return used_tables() == 0; } > bool walk(Item_processor processor, bool walk_subquery, uchar *arg) > { > return (*ref)->walk(processor, walk_subquery, arg) || > > === modified file 'sql/table.cc' > --- a/sql/table.cc 2014-10-29 10:22:48 +0000 > +++ b/sql/table.cc 2014-12-19 13:23:50 +0000 The function get_real_join_table() lacks comments, which makes it difficult to review. Do I understand correctly that its action can be described by this comment: /* @brief Given a merged view (or a merged derived table), find its first* base** table. @detail (*) first means as defined in the syntax (**) base means we need a physical table, one that has a bit in table->map and is present in select_lex->leaf_tables. Example: t1 join ( (select * from t2, t3 ) as derivedA join (select * from t5, t6) as derivedC ) as derivedB here, derivedB->get_real_join_table()="t2" */ > @@ -5012,7 +5012,8 @@ TABLE *TABLE_LIST::get_real_join_table() > TABLE_LIST *tbl= this; > while (tbl->table == NULL || tbl->table->reginfo.join_tab == NULL) I'm quite surprised that joins are run over tables that don't have reginfo.join_tab set appropriately... > { > - if (tbl->view == NULL && tbl->derived == NULL) > + if ((tbl->view == NULL && tbl->derived == NULL) || > + tbl->is_materialized_derived()) I'm wondering why part of conditions are in while(...) and the other part is here. > break; > /* we do not support merging of union yet */ > DBUG_ASSERT(tbl->view == NULL || > The code below this line looks bizarre (and it is still from your recent fixes to VIEWs): { List_iterator_fast<TABLE_LIST> ti; { List_iterator_fast<TABLE_LIST> ti(tbl->view != NULL ? why define 'ti' and then immediately redefine it? BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

2 1

Re: [Maria-developers] f954bb7: MDEV-7671: Cache VIEW definitions in the TDC
by Sergei Golubchik 11 Mar '15

11 Mar '15

Hi, Sanja! On Mar 10, sanja(a)mariadb.com wrote: > revision-id: f954bb7fdc527cf816253fe0e7adb4239c368624 > parent(s): f7d196bb2e605766668895027c48728f03803734 > committer: Oleksandr Byelkin > branch nick: server > timestamp: 2015-03-10 12:52:45 +0100 > message: > > MDEV-7671: Cache VIEW definitions in the TDC > > (changes of backported patch are very small: strlen removed, error > processing fixed, view open statistics added) Ok to push Regards, Sergei

1 0

[Maria-developers] GSOC 2015: Support for popular formats on Dynamic columns.
by arlene seuleu 11 Mar '15

11 Mar '15

Hi I am Arlene Flavie a software Engineering student and I wish to make my contribution to opensource through Mariadb. I have build mariadb from a git clone and ran the tests. I am interested in working on adding routines to import and export popular data formats to and from dynamic columns. I am currently studying the dynamic column data structures in mysql/ma_dyncol.** and I think what Oleksandr Byeklin proposed makes sense to me. Thats using methods like xxx_init() and xxx_named() to play with the old dynamic column blob format. I will be applying to work on this task for Gsoc 2015 and im hoping to learn much in open source development. Cheers.

2 1

[Maria-developers] Interest in GSoc
by Acha Bill 11 Mar '15

11 Mar '15

Hello, I am Acha Bill, Computer Engineering, University of Buea, Cameroon. Second year undergraduate. I am very much interested in the project; Import and export popular data formats from and to dynamic columns. for Gsoc 15. I have experience with C. Any pointers as to what next will be appreciated. Thanks.

2 1

Re: [Maria-developers] [Maria-discuss] Enabling feedback pluging for MariaDB 10.1.4
by Justin Swanhart 11 Mar '15

11 Mar '15

It tells you how much a sample used and you don't know what fraction of the total that represents, or if their usage is representative of the larger base. I maintain they are not meaningful but you can feel free to disagree. Sent from my iPhone > On Mar 10, 2015, at 7:42 PM, Michael Widenius <michael.widenius(a)gmail.com> wrote: > > Hi! > > <cut> > >> I agree with Kristian. Given the way it works, the statistics are really meaningless and I feel you shouldn't drive important choices based on bad statistics. > > From a statistics point of view, the automatic feedback will tell us a > lot about how people use MariaDB and is far from meaningless! > (We only need a small statistical sample to know how the majority is > using the server). > > The most important things for our developers, that the phone home > feature will tell us, are probably: > - Features that are used (and how much they are used) > - Features that not used. > - Combinations of features used. > - The stability of the servers. > - How often upgrades are done > > This will allow us to better concentrate the few development resources > on features that are important to our users. > > In the past, our main problem for allocating resources has always been > "does anyone really use or care about this". Now we finally will get > some information to make these kind of decisions easier. > > Regards, > Monty

2 1

[Maria-developers] GSOC 2015 :UDFs returning arrays
by Blessing Atie 10 Mar '15

10 Mar '15

Hi My name is Blessing Atie and i would love to participate in the Google summer of code 2015 under Mariadb. I am interested in the project UDF returning arrays. I have build mariadb from source and and i also ran the test suites. I am currently studing the sql/sql.** files to understand UDF's and i also downloaded some ebook on mysql internals and mysql plugin development which im reading while hoping to find a breakthrough on this project in them. I would love to know if there are any bugs related to this task on JIRA I could work on as a means of preparing for the project. Please give me the link. Regards Blessing.

1 0

Re: [Maria-developers] [Commits] 83659a6: MDEV-7006 MDEV-7007: SET STATEMENT and slow log
by Sergei Golubchik 10 Mar '15

10 Mar '15

Hi, Sanja! On Feb 17, sanja(a)mariadb.com wrote: > revision-id: 83659a68642da2eb6e9327f17a0ef730f512aa96 > parent(s): 3e2849d2a01b06a61407b00989c3f981e62dd183 > committer: Oleksandr Byelkin > branch nick: server > timestamp: 2015-02-17 12:54:51 +0100 > message: > > MDEV-7006 MDEV-7007: SET STATEMENT and slow log > fixed embedded server tests > MDEV-7009: SET STATEMENT min_examined_row_limit has no effect > MDEV-6948:SET STATEMENT gtid_domain_id = ... FOR has no effect (same for gtid_seq_no and server_id) > > old values of SET STATENENT variables now saved in its own Query_arena and restored later Looks good to me. I didn't check, though, whether you've put every restore correctly. See minor comments below: > diff --git a/mysql-test/t/set_statement_notembedded_binlog.test b/mysql-test/t/set_statement_notembedded_binlog.test > new file mode 100644 > index 0000000..c9c2334 > --- /dev/null > +++ b/mysql-test/t/set_statement_notembedded_binlog.test > @@ -0,0 +1,22 @@ > +--source include/have_log_bin.inc > +--source include/not_embedded.inc > + > +--disable_warnings > +drop table if exists t1; > +drop view if exists t1; > +--enable_warnings This was useful in our early MySQL days, but it's not anymore. I've stopped adding these drop...if exists at the beginning of test files a couple of years ago. mtr has a after-test check to ensure that no test leaves its tables behind. So one can rely on every test starting clean. > + > +--echo # > +--echo # MDEV-6948: SET STATEMENT gtid_domain_id = ... FOR has no effect > +--echo # (same for gtid_seq_no and server_id) > +--echo # > +reset master; > +create table t1 (i int); > +set gtid_domain_id = 10; > +insert into t1 values (1),(2); > +set statement gtid_domain_id = 20 for insert into t1 values (3),(4); > + > +--replace_column 1 x 2 x 3 x 4 x 5 x > +show binlog events limit 5,5; > + > +drop table t1; > diff --git a/sql/sql_lex.cc b/sql/sql_lex.cc > index 0c1c3b3..ee956ed 100644 > --- a/sql/sql_lex.cc > +++ b/sql/sql_lex.cc > @@ -4223,6 +4224,56 @@ int LEX::print_explain(select_result_sink *output, uint8 explain_flags, > return res; > } > > +void LEX::set_arena_for_set_stmt(Query_arena *backup) > +{ > + DBUG_ENTER("LEX::set_arena_for_set_stmt"); > + DBUG_ASSERT(arena_for_set_stmt== 0); > + if (!mem_root_for_set_stmt) > + { > + mem_root_for_set_stmt= new MEM_ROOT(); > + if (!(mem_root_for_set_stmt)) > + DBUG_VOID_RETURN; What happens in OOM case? You return early, but does the caller know that? Or it'll continue, thinking that everything's ok? > + init_sql_alloc(mem_root_for_set_stmt, ALLOC_ROOT_SET, ALLOC_ROOT_SET, > + MYF(MY_THREAD_SPECIFIC)); > + } > + if (!(arena_for_set_stmt= new Query_arena(mem_root_for_set_stmt, Why are you allocating memory for Query_arena, instead of creating it on mem_root_for_set_stmt? > + Query_arena::STMT_INITIALIZED))) > + DBUG_VOID_RETURN; > + DBUG_PRINT("info", ("mem_root: 0x%lx arena: 0x%lx", > + (ulong) mem_root_for_set_stmt, > + (ulong) arena_for_set_stmt)); > + thd->set_n_backup_active_arena(arena_for_set_stmt, backup); > + DBUG_VOID_RETURN; > +} > + > + > +void LEX::reset_arena_for_set_stmt(Query_arena *backup) > +{ > + DBUG_ENTER("LEX::reset_arena_for_set_stmt"); > + DBUG_ASSERT(arena_for_set_stmt); > + thd->restore_active_arena(arena_for_set_stmt, backup); > + DBUG_PRINT("info", ("mem_root: 0x%lx arena: 0x%lx", > + (ulong) arena_for_set_stmt->mem_root, > + (ulong) arena_for_set_stmt)); > + DBUG_VOID_RETURN; > +} > + > + > +void LEX::free_arena_for_set_stmt() > +{ > + DBUG_ENTER("LEX::free_arena_for_set_stmt"); > + if (!arena_for_set_stmt) > + return; > + DBUG_PRINT("info", ("mem_root: 0x%lx arena: 0x%lx", > + (ulong) arena_for_set_stmt->mem_root, > + (ulong) arena_for_set_stmt)); > + arena_for_set_stmt->free_items(); > + delete(arena_for_set_stmt); > + free_root(mem_root_for_set_stmt, MYF(MY_KEEP_PREALLOC)); > + arena_for_set_stmt= 0; > + DBUG_VOID_RETURN; > +} > + > void LEX::restore_set_statement_var() > { > DBUG_ENTER("LEX::restore_set_statement_var"); > diff --git a/sql/sql_lex.h b/sql/sql_lex.h > index 46d667c..ed3c4a0 100644 > --- a/sql/sql_lex.h > +++ b/sql/sql_lex.h > @@ -2677,10 +2688,22 @@ struct LEX: public Query_tables_list > limit_rows_examined_cnt= ULONGLONG_MAX; > } > > + > + inline void free_set_stmt_mem_root() > + { Please, add DBUG_ASSERT(!is_arena_for_set_stmt()); > + if (mem_root_for_set_stmt) > + { > + free_root(mem_root_for_set_stmt, MYF(0)); > + delete mem_root_for_set_stmt; > + mem_root_for_set_stmt= 0; > + } > + } > + > LEX(); > > virtual ~LEX() > { > + free_set_stmt_mem_root(); > destroy_query_tables_list(); > plugin_unlock_list(NULL, (plugin_ref *)plugins.buffer, plugins.elements); > delete_dynamic(&plugins); Regards, Sergei

2 1

[Maria-developers] GSOC 2015: Indexes on virtual columns
by Sichom Lionel 10 Mar '15

10 Mar '15

Hello, My name is Lionel Sichom, 2nd year undergraduate student at the University of Buea. I am a newbie in Open source community and I am interested in participating at the Google Summer of Code 2015 with MariaDB. I am mostly interested with the project Indexes on virtual columns. As for now I have 5.5.41-MariaDB version and I have already downloded the code, and I am already compiling the code. Can someone please guide me through the steps next to follow. Thanks Lionel Sichom

2 1

[Maria-developers] Question regarding MDEV-7682
by Vicențiu Ciorbaru 10 Mar '15

10 Mar '15

Hi Holyfoot! I've made some changes regarding key_length computation to fix MDEV-6838. Changing this seems to have lead to us discovering another issue outlined in MDEV-7682 (that I've just created). It seems that the spatial key returns a key_length of 0. I've spoken to Sergei Petrunia and we are not sure about this particular case. pack_length returns a bigger number (12). According to the other comments in the code, pack_length also returns the length field associated with the column, but we are only interested in the length of the column field, without any metadata information. The function TABLE:create_key_part_by_field adds the metadata length itself afterwards. Do you have any input on this issue, namely why the key_length is returned as 0 for the LINESTRING column? Regards, Vicentiu

2 1

[Maria-developers] [GSoC'15] Running MaxScale and MariaDB servers on single machine
by sriram patil 09 Mar '15

09 Mar '15

Hi All, I tried running 2 MariaDB servers and MaxScale on my laptop. I gave different configurations to both MariaDB servers with separate data and log directories. Referred this <http://www.ducea.com/2009/01/19/running-multiple-instances-of-mysql-on-the-…> for it. Also have set up MaxScale from "MaxScale for rest of us" series on MariaDB blog. 1. https://mariadb.com/blog/maxscale-rest-us-part-1 2. https://mariadb.com/blog/maxscale-rest-us-part-2 3. https://mariadb.com/blog/maxscale-rest-us-part-3 4. https://mariadb.com/blog/maxscale-rest-us-part-4 So, i tried to set up the readwritesplit router with 1 master and 1 slave. Here, the replication mechanism worked fine but the queries are always executed on the master server. Example output for a read query: mysql -h localhost -u maxuser -p test -e "select c1, @@server_id from t1" Enter password: +------+-------------+ | c1 | @@server_id | +------+-------------+ | 1 | 1 | +------+-------------+ server_id for master is 1 whereas for slave it is 11, which is never displayed in the output. Can someone help me with this? Should I try the same config with one more slave? Are there any better resources for setting up MaxScale? Thanks, Sriram

2 5

[Maria-developers] MariaDB Maxscale Project (GSOC 2015)
by Aalekh Nigam 09 Mar '15

09 Mar '15

Hello, I am Aalekh Nigam, 3rd year Undergraduate Student at JIIT, Noida. I have recently successfully completed Mozilla Winter Of Security and Owasp Winter of Code, and is aiming for MaxScale projects for this GSOC. Can someone please guide me through starter’s bug for Maxscale to get my hands on with MaxScale development ? Thank You —Aalekh

2 1

[Maria-developers] MDEV-7643 review
by Elena Stepanova 09 Mar '15

09 Mar '15

Hi Sergei, http://lists.askmonty.org/pipermail/commits/2015-March/007508.html Could you please check patch for https://mariadb.atlassian.net/browse/MDEV-7643 (MTR creates nested links when tests are run with --mem)? It's in the area that you looked at recently, so you might think of something that I could have missed. Thanks, Elena

2 1

Re: [Maria-developers] [Commits] e464bc6: MDEV-7107 Sporadic test failure in multi_source.multisource
by Kristian Nielsen 09 Mar '15

09 Mar '15

elenst(a)montyprogram.com writes: > revision-id: e464bc6fe3d7566831da5e26447d90e758b5aa2a > committer: Elena Stepanova > timestamp: 2015-03-09 02:07:47 +0200 > message: > > MDEV-7107 Sporadic test failure in multi_source.multisource > > Extend show_slave_status.inc to run SHOW ALL SLAVES STATUS and > SHOW SLAVE 'name' STATUS on demand, and make the test use > the include file instead of direct SHOW statements Looks fine, thanks! Ok to push. I agree with your changes to show_slave_status.inc. I think they will be useful for futute test cases also. - Kristian.

1 0

[Maria-developers] MDEV-7107 review
by Elena Stepanova 09 Mar '15

09 Mar '15

Hi Kristian, Could you please take a quick look at the patch for MDEV-7107 (Sporadic test failure in multi_source.multisource). http://lists.askmonty.org/pipermail/commits/2015-March/007543.html I don't expect you'll mind changes to the test itself, we discussed earlier that's how SHOW SLAVE STATUS should be handled; but please check if you're okay with the additions to show_slave_status.inc, or you would prefer a separate include file for SHOW ALL SLAVES STATUS instead. Thanks, Elena

1 0

[Maria-developers] rpl test with gtid as default
by Jonas Oreland 09 Mar '15

09 Mar '15

what do you think ? /Jonas

3 3

Re: [Maria-developers] a1c4620: MDEV-6858: enforce_storage_engine option
by Sergei Golubchik 09 Mar '15

09 Mar '15

Hi, Jan! On Mar 08, Jan Lindström wrote: > revision-id: a1c4620646d43eaff979c132a57ee2101070fa5e > parent(s): 1b74f32b1d94465f6d6e14e4aa7ba1f94c32e39e > committer: Jan Lindström > branch nick: 10.1-innodb > timestamp: 2015-03-08 17:23:05 +0200 > message: > > MDEV-6858: enforce_storage_engine option > > Merge from Percona Server enforced use of a specific storage engine > authored by Stewart Smith. > > Modified to be session variable and modifiable only by SUPER. Use > similar implementation as default_storage_engine. > > diff --git a/mysql-test/t/enforce_storage_engine.test b/mysql-test/t/enforce_storage_engine.test > new file mode 100644 > index 0000000..36231e6 > --- /dev/null > +++ b/mysql-test/t/enforce_storage_engine.test > @@ -0,0 +1,35 @@ ... > +--error ER_UNKNOWN_STORAGE_ENGINE > +CREATE TABLE t2 (c1 INT PRIMARY KEY AUTO_INCREMENT, c2 VARCHAR(10)) ENGINE=MyISAM; > + > +SET SESSION enforce_storage_engine=MyISAM; > +select @@session.enforce_storage_engine; > +select * from t1; > +drop table t1; > + > +--error 1286 > +SET SESSION enforce_storage_engine=FooBar; > + > +select @@session.enforce_storage_engine; Consider adding tests for "only modifiable by SUPER" (create a new user, connect with it, see that enforce_storage_engine cannot be changed), and for modified enforce_storage_engine (you tested that it can be changed, but didn't do any CREATE TABLE with the new value). > diff --git a/sql/handler.cc b/sql/handler.cc > index 12d7ffb..5d5ac30 100644 > --- a/sql/handler.cc > +++ b/sql/handler.cc > @@ -117,6 +117,15 @@ static plugin_ref ha_default_tmp_plugin(THD *thd) > return ha_default_plugin(thd); > } > > + > +static plugin_ref ha_enforced_plugin(THD *thd) > +{ > + if (thd->variables.enforced_table_plugin) > + return thd->variables.enforced_table_plugin; > + return NULL; > +} > + > + > /** @brief > Return the default storage engine handlerton for thread > > @@ -148,6 +157,25 @@ handlerton *ha_default_tmp_handlerton(THD *thd) > > > /** @brief > + Return the enforced storage engine handlerton for thread > + > + SYNOPSIS > + ha_enforce_handlerton(thd) > + thd current thread > + > + RETURN > + pointer to handlerton OR NULL > + > +*/ > +handlerton *ha_enforced_handlerton(THD *thd) > +{ > + plugin_ref plugin= ha_enforced_plugin(thd); > + if (plugin) > + return plugin_hton(plugin); > + return NULL; > +} > + > +/** @brief > Return the storage engine handlerton for the supplied name > > SYNOPSIS > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index b97742d..6966fd9 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -4738,8 +4738,8 @@ static void add_file_to_crash_report(char *file) > #define init_default_storage_engine(X,Y) \ > init_default_storage_engine_impl(#X, X, &global_system_variables.Y) > > -static int init_default_storage_engine_impl(const char *opt_name, > - char *engine_name, plugin_ref *res) > +int init_default_storage_engine_impl(const char *opt_name, > + char *engine_name, plugin_ref *res) You've made init_default_storage_engine_impl() extern, but you never use it anywhere. And you've forgot to create a command line option for enforced_table_plugin. > { > if (!engine_name) > { > diff --git a/sql/sql_table.cc b/sql/sql_table.cc > index e9a1ae9..184c073 100644 > --- a/sql/sql_table.cc > +++ b/sql/sql_table.cc > @@ -9745,12 +9745,24 @@ static bool check_engine(THD *thd, const char *db_name, > DBUG_ENTER("check_engine"); > handlerton **new_engine= &create_info->db_type; > handlerton *req_engine= *new_engine; > + handlerton *enf_engine= ha_enforced_handlerton(thd); I'd write it directly as thd->variables.enforced_table_plugin ? plugin_hton(thd->variables.enforced_table_plugin) : NULL; Or as an inlined function here, in sql_table.cc. Not as two small but non-inlined functions in handler.cc, that are only used once here. > bool no_substitution= thd->variables.sql_mode & MODE_NO_ENGINE_SUBSTITUTION; > *new_engine= ha_checktype(thd, req_engine, no_substitution); > DBUG_ASSERT(*new_engine); > if (!*new_engine) > DBUG_RETURN(true); > > + if (enf_engine) > + { > + if (enf_engine != *new_engine && no_substitution) > + { > + const char *engine_name= ha_resolve_storage_engine_name(req_engine); > + my_error(ER_UNKNOWN_STORAGE_ENGINE, MYF(0), engine_name, engine_name); > + DBUG_RETURN(TRUE); > + } > + *new_engine= enf_engine; > + } > + > if (req_engine && req_engine != *new_engine) > { > push_warning_printf(thd, Sql_condition::WARN_LEVEL_NOTE, > diff --git a/sql/sys_vars.cc b/sql/sys_vars.cc > index 8371df0..cd53714 100644 > --- a/sql/sys_vars.cc > +++ b/sql/sys_vars.cc > @@ -3437,6 +3437,28 @@ static Sys_var_plugin Sys_default_tmp_storage_engine( > SESSION_VAR(tmp_table_plugin), NO_CMD_LINE, > MYSQL_STORAGE_ENGINE_PLUGIN, DEFAULT(&default_tmp_storage_engine)); > > +extern int init_default_storage_engine_impl(const char* opt_name, char *engine_name, plugin_ref *plugin); Not used below. > + > + > +static bool check_super_and_not_null(sys_var *self, THD*thd, set_var *var) I think that NULL value should be allowed, it means "don't enforce a storage engine". > +{ > +#ifndef NO_EMBEDDED_ACCESS_CHECKS > + if (!(thd->security_ctx->master_access & SUPER_ACL)) > + { > + my_error(ER_SPECIFIC_ACCESS_DENIED_ERROR, MYF(0), "SUPER"); > + return true; > + } > +#endif > + return var->value && var->value->is_null(); > +} > + > +static Sys_var_plugin Sys_enforce_storage_engine( > + "enforce_storage_engine", "Force the use of a storage engine for new " > + "tables", > + SESSION_VAR(enforced_table_plugin), > + NO_CMD_LINE, MYSQL_STORAGE_ENGINE_PLUGIN, > + DEFAULT(0), NO_MUTEX_GUARD, NOT_IN_BINLOG, ON_CHECK(check_super_and_not_null)); > + > #if defined(ENABLED_DEBUG_SYNC) > /* > Variable can be set for the session only. Regards, Sergei

1 0

[Maria-developers] [GSoC'15] MaxScale Project
by sriram patil 07 Mar '15

07 Mar '15

Hi Markus, I am Sriram, a Masters' student at IIIT Hyderabad, India. I have worked with MariaDB in GSoC-2014 on MDEV-5359 under Alexander Barkov. I have also documented my work on my blog <http://srirampatil.tumblr.com/>. I am interested to work on "MaxScale filter to capture incoming operations for consumption in external sources". I have downloaded and built MariaDB 10.1 source code from github. I was trying to build MaxScale but it resulted in some errors. I followed the installation instructions given on MaxScale github(develop branch). The error is as follows /home/sriram/gsoc/MaxScale/query_classifier/query_classifier.cc: In function ‘bool skygw_stmt_causes_implicit_commit(LEX*, int*)’: /home/sriram/gsoc/MaxScale/query_classifier/query_classifier.cc:848:40: error: ‘struct LEX’ has no member named ‘drop_temporary’ succp = !(lex->drop_temporary); ^ make[2]: *** [query_classifier/CMakeFiles/query_classifier.dir/query_classifier.cc.o] Error 1 make[1]: *** [query_classifier/CMakeFiles/query_classifier.dir/all] Error 2 I checked struct LEX from MariaDB source code and it does not have the mentioned field. So, is MaxScale expecting some other MariaDB source code? It would be great if you could give me some pointers regarding the same. Thanks, Sriram Patil

2 2

Re: [Maria-developers] GSoC 15 : MaxScale Project under Markus Makela
by Markus Mäkelä 06 Mar '15

06 Mar '15

Hi, The most up to date documentation for MaxScale can be found on the develop branch on GitHub: https://github.com/mariadb-corporation/MaxScale/blob/develop/Documentation/… I suggest using that rather than the PDF documentation since it's a lot more easy to use. The filter interface can be found in the server/include/filter.h file. All filters must implement these functions for them to work with MaxScale. The filters are located in the server/modules/filter folder. The best way to get a hang of the filters is to take a look at one and the regexfilter and qlafilter are the ones that show the filter interface in action without having much code in them. Other useful stuff to look at can be found in the server/include folder are buffer.h(MaxScale uses these to route queries though it), modutils.h(useful functions for SQL handling), housekeeper.h(for tasks that take a long time this can be used) and hashtable.h(a generic hashtable). All of the MaxScale team works around UTC+1. For quick questions ask them on #maxscale on Freenode but email will most likely be answered as fast as IRC. Markus > Hello guys, > > As per the official list > <https://mariadb.com/kb/en/mariadb/google-summer-of-code-2015/> of projects > by MariaDB, I'm interested in the project - MaxScale filter to capture > incoming operations for consumption in external sources ( Mentor : Markus > Makela ) > > *I've 3 requests/doubts : * > > 0.I've downloaded the MaxScale code from github > <https://github.com/mariadb-corporation/MaxScale> and have read the > documentation. > It says the filter configuration file is : MaxScale.ini > I didn't find this file in the codebase. > > Nor was I able to figure out the code-location related to the "filter" > module. > > 1.Kindly suggest for the further things I should do/try. > > Shall I install MariaDB & MaxScale? > What part of code-base should I look into & play around? > > 2.What is the UTC time when most of you guys are online on IRC-channel > (#maria)? > > Since I'm from India, I need to know the time when most of you guys are > online to get immediate help on IRC. > > > Thanks in advance. > > -- > Lokesh Walase -- Markus Mäkelä, Software Engineer MariaDB Corporation t: +358 40 7740484 | Skype: markus.j.makela

1 0

[Maria-developers] Comments/thoughts on patch
by Jonas Oreland 06 Mar '15

06 Mar '15

Hi Kristian, I wrote attached patch (so that i would understand what code actually does) and found myself while refactoring also doing a change, namely that i removed the check for server_id in a couple of places. it still passes all tests... could you comment on this, or write(outline) a testcase that fails with my patch. i think functionality that i might have removed would be if you write have 2 mysqld M1 & M2 and your replicate both way...and you *don't* use domain_id...and you write concurrently on both mysqld (but maybe disjoint data ??). feedback/comments appreciated. /Jonas ps. patch applies to f4f37533a09b9776e8d5ac3f3a27957f553c9043

2 9

[Maria-developers] MaxScale JIRA project
by Vangelis Katsikaros 05 Mar '15

05 Mar '15

Hi While browsing the GSOC projects, I noticed that the MX project, and the tickets mentioned, require signing in. I don't know if this intentional or not (the MDEV project doesn't require signin for viewing the project). Just thought I should mention it. Regards Vangelis

2 2

[Maria-developers] GSoC 15 : MaxScale Project under Markus Makela
by Lokesh Walase 04 Mar '15

04 Mar '15

Hello guys, As per the official list <https://mariadb.com/kb/en/mariadb/google-summer-of-code-2015/> of projects by MariaDB, I'm interested in the project - MaxScale filter to capture incoming operations for consumption in external sources ( Mentor : Markus Makela ) *I've 3 requests/doubts : * 0.I've downloaded the MaxScale code from github <https://github.com/mariadb-corporation/MaxScale> and have read the documentation. It says the filter configuration file is : MaxScale.ini I didn't find this file in the codebase. Nor was I able to figure out the code-location related to the "filter" module. 1.Kindly suggest for the further things I should do/try. Shall I install MariaDB & MaxScale? What part of code-base should I look into & play around? 2.What is the UTC time when most of you guys are online on IRC-channel (#maria)? Since I'm from India, I need to know the time when most of you guys are online to get immediate help on IRC. Thanks in advance. -- Lokesh Walase

1 0

[Maria-developers] MDEV-6838: Using too big key for internal temp tables
by Vicențiu Ciorbaru 03 Mar '15

03 Mar '15

Hi Sergei and Sergey! I've implemented the fix for MDEV-6838. I'd like a review from both of you regarding the patch. There are a couple of changes made. The main issues are outlined in the commit message. I've changed the way create_key_part_by_field gets called in order to decouple the KEY_PART_INFO creation process from any KEYINFO modifications and also documented what the method does. My other concern is that create_key_part_by_field does not seem to make any sense within the table class as it's basically a constructor for KEY_PART_INFO and makes no use of table fields. I suggest we move the code as either a constructor for KEY_PART_INFO, or an init() function. I think that would be better left as a separate patch, but the method itself is very easy to factor out in case you agree with the change. I'm looking forward for your input. Regards, Vicențiu ---------- Forwarded message ---------- From: <vicentiu(a)mariadb.org> Date: 28 February 2015 at 23:58 Subject: 45b6edb: MDEV-6838: Using too big key for internal temp tables To: commits(a)mariadb.org revision-id: 45b6edb158f8101d641f550179ee15df363f686f parent(s): fa87fc733d7855e0e5f9b959ca0bddf772ca57e5 committer: Vicențiu Ciorbaru branch nick: server timestamp: 2015-02-28 23:58:05 +0200 message: MDEV-6838: Using too big key for internal temp tables This bug manifests due to wrong computation and evaluation of keyinfo->key_length. The issues were: * Using table->file->max_key_length() as an absolute value that must not be reached for a key, while it represents the maximum number of bytes possible for a table key. * Incorrectly computing the keyinfo->key_length size during KEY_PART_INFO creation. The metadata information regarding the key such the field length (for strings) was added twice. --- mysql-test/r/table_keyinfo-6838.result | 12 ++++++++++++ mysql-test/t/table_keyinfo-6838.test | 18 ++++++++++++++++++ sql/sql_select.cc | 7 ++++--- sql/structs.h | 1 + sql/table.cc | 34 ++++++++++++++++++++++++++-------- sql/table.h | 2 +- 6 files changed, 62 insertions(+), 12 deletions(-) diff --git a/mysql-test/r/table_keyinfo-6838.result b/mysql-test/r/table_keyinfo-6838.result new file mode 100644 index 0000000..55b0350 --- /dev/null +++ b/mysql-test/r/table_keyinfo-6838.result @@ -0,0 +1,12 @@ +CREATE TABLE t1 (i INT, state VARCHAR(997)) ENGINE=MyISAM; +INSERT INTO t1 VALUES (2,'Louisiana'),(9,'Maine'); +CREATE TABLE t2 (state VARCHAR(997), j INT) ENGINE=MyISAM; +INSERT INTO t2 VALUES ('Louisiana',9),('Alaska',5); +INSERT INTO t2 SELECT t2.* FROM t2 JOIN t2 AS t3 JOIN t2 AS t4 JOIN t2 AS t5 JOIN t2 AS t6; +SET @@max_heap_table_size= 16384; +set @@optimizer_switch='derived_merge=OFF'; +set @@optimizer_switch='extended_keys=ON'; +SELECT * FROM t1 AS t1_1 LEFT JOIN ( t1 AS t1_2 INNER JOIN (SELECT * FROM t2) v2 ON t1_2.i = j ) ON t1_1.state = v2.state LIMIT 1; +i state i state state j +2 Louisiana 9 Maine Louisiana 9 +DROP TABLE t1, t2; diff --git a/mysql-test/t/table_keyinfo-6838.test b/mysql-test/t/table_keyinfo-6838.test new file mode 100644 index 0000000..5f4c9f4 --- /dev/null +++ b/mysql-test/t/table_keyinfo-6838.test @@ -0,0 +1,18 @@ +# Test case for MDEV-6838. +# Due to incorrect key_length computation, this usecase failed with the error +# Using too big key for internal temp table. + +CREATE TABLE t1 (i INT, state VARCHAR(997)) ENGINE=MyISAM; +INSERT INTO t1 VALUES (2,'Louisiana'),(9,'Maine'); + +CREATE TABLE t2 (state VARCHAR(997), j INT) ENGINE=MyISAM; +INSERT INTO t2 VALUES ('Louisiana',9),('Alaska',5); +INSERT INTO t2 SELECT t2.* FROM t2 JOIN t2 AS t3 JOIN t2 AS t4 JOIN t2 AS t5 JOIN t2 AS t6; + +SET @@max_heap_table_size= 16384; +set @@optimizer_switch='derived_merge=OFF'; +set @@optimizer_switch='extended_keys=ON'; + +SELECT * FROM t1 AS t1_1 LEFT JOIN ( t1 AS t1_2 INNER JOIN (SELECT * FROM t2) v2 ON t1_2.i = j ) ON t1_1.state = v2.state LIMIT 1; + +DROP TABLE t1, t2; diff --git a/sql/sql_select.cc b/sql/sql_select.cc index 387fdb3..dabe46e 100644 --- a/sql/sql_select.cc +++ b/sql/sql_select.cc @@ -7957,7 +7957,8 @@ static bool create_hj_key_for_table(JOIN *join, JOIN_TAB *join_tab, { Field *field= table->field[keyuse->keypart]; uint fieldnr= keyuse->keypart+1; - table->create_key_part_by_field(keyinfo, key_part_info, field, fieldnr); + table->create_key_part_by_field(key_part_info, field, fieldnr); + keyinfo->key_length += key_part_info->store_length; key_part_info++; } } @@ -15921,7 +15922,7 @@ bool create_internal_tmp_table(TABLE *table, KEY *keyinfo, goto err; bzero(seg, sizeof(*seg) * keyinfo->key_parts); - if (keyinfo->key_length >= table->file->max_key_length() || + if (keyinfo->key_length > table->file->max_key_length() || keyinfo->key_parts > table->file->max_key_parts() || share->uniques) { @@ -16107,7 +16108,7 @@ bool create_internal_tmp_table(TABLE *table, KEY *keyinfo, goto err; bzero(seg, sizeof(*seg) * keyinfo->key_parts); - if (keyinfo->key_length >= table->file->max_key_length() || + if (keyinfo->key_length > table->file->max_key_length() || keyinfo->key_parts > table->file->max_key_parts() || share->uniques) { diff --git a/sql/structs.h b/sql/structs.h index ae71819..9840cec 100644 --- a/sql/structs.h +++ b/sql/structs.h @@ -76,6 +76,7 @@ typedef struct st_key_part_info { /* Info about a key part */ */ uint16 store_length; uint16 key_type; + /* Fieldnr begins counting from 1 */ uint16 fieldnr; /* Fieldnum in UNIREG */ uint16 key_part_flag; /* 0 or HA_REVERSE_SORT */ uint8 type; diff --git a/sql/table.cc b/sql/table.cc index cfa0b6c..7df2ae6 100644 --- a/sql/table.cc +++ b/sql/table.cc @@ -5968,18 +5968,32 @@ bool TABLE::alloc_keys(uint key_count) } -void TABLE::create_key_part_by_field(KEY *keyinfo, - KEY_PART_INFO *key_part_info, +/** + @brief + Populate a KEY_PART_INFO structure with the data related to a field entry. + + @param key_part_info The structure to fill. + @param field The field entry that represents the key part. + @param fleldnr The number of the field, count starting from 1. + + TODO: This method does not make use of any table specific fields. It + could be refactored to act as a constructor for KEY_PART_INFO instead. +*/ + +void TABLE::create_key_part_by_field(KEY_PART_INFO *key_part_info, Field *field, uint fieldnr) -{ +{ key_part_info->null_bit= field->null_bit; key_part_info->null_offset= (uint) (field->null_ptr - (uchar*) record[0]); key_part_info->field= field; key_part_info->fieldnr= fieldnr; key_part_info->offset= field->offset(record[0]); - key_part_info->length= (uint16) field->pack_length(); - keyinfo->key_length+= key_part_info->length; + /* + field->key_length() accounts for the raw length of the field, excluding + any metadata such as length of field or the NULL flag. + */ + key_part_info->length= (uint16) field->key_length(); key_part_info->key_part_flag= 0; /* TODO: The below method of computing the key format length of the @@ -5991,17 +6005,20 @@ void TABLE::create_key_part_by_field(KEY *keyinfo, */ key_part_info->store_length= key_part_info->length; + /* + The total store length of the key part is the raw length of the field + + any metadata information, such as its length for strings and/or the null + flag. + */ if (field->real_maybe_null()) { key_part_info->store_length+= HA_KEY_NULL_LENGTH; - keyinfo->key_length+= HA_KEY_NULL_LENGTH; } if (field->type() == MYSQL_TYPE_BLOB || field->type() == MYSQL_TYPE_GEOMETRY || field->real_type() == MYSQL_TYPE_VARCHAR) { key_part_info->store_length+= HA_KEY_BLOB_LENGTH; - keyinfo->key_length+= HA_KEY_BLOB_LENGTH; // ??? key_part_info->key_part_flag|= field->type() == MYSQL_TYPE_BLOB ? HA_BLOB_PART: HA_VAR_LENGTH_PART; } @@ -6124,7 +6141,8 @@ bool TABLE::add_tmp_key(uint key, uint key_parts, if (key_start) (*reg_field)->key_start.set_bit(key); (*reg_field)->part_of_key.set_bit(key); - create_key_part_by_field(keyinfo, key_part_info, *reg_field, fld_idx+1); + create_key_part_by_field(key_part_info, *reg_field, fld_idx+1); + keyinfo->key_length += key_part_info->store_length; (*reg_field)->flags|= PART_KEY_FLAG; key_start= FALSE; key_part_info++; diff --git a/sql/table.h b/sql/table.h index 7a1e380..8324454 100644 --- a/sql/table.h +++ b/sql/table.h @@ -1269,7 +1269,7 @@ struct TABLE bool add_tmp_key(uint key, uint key_parts, uint (*next_field_no) (uchar *), uchar *arg, bool unique); - void create_key_part_by_field(KEY *keyinfo, KEY_PART_INFO *key_part_info, + void create_key_part_by_field(KEY_PART_INFO *key_part_info, Field *field, uint fieldnr); void use_index(int key_to_save); void set_table_map(table_map map_arg, uint tablenr_arg)

2 1

[Maria-developers] MDEV-6089 review
by Sergei Golubchik 03 Mar '15

03 Mar '15

Hi. Here're I'll be asking questions about the patches I'm looking at. 1. Is this mostly a merge of WL#7305 or mostly your own implementation? That is, most of the changes and design decisions are yours or "because MySQL did it"? 2. Do you work like I do (commit, push, fix failing tests, combine commits with "git rebase -i" to get a clean history) or you've just magically got 32bit rdiffs correctly in every commit? :) 3. Why "MDL objects cache won't be needed"? The comment said "On some systems (e.g. Windows XP) constructing/destructing MDL_lock objects can be fairly expensive. We use this cache to avoid these costs..." Why is this no longer an issue? 4. MDL_lock_strategy - this object is in 5.7 too, but you've done it very differently. Why? And I wonder why didn't you move the common part of two strategies to the base class. You wanted a pure interface class? 5. Added initializer callback to lf-hash. May be you should remove this hack with decreasing LF_HASH::element_size etc and move this functionality to a custom initializer? It was added in 6ba12f. In particular, see changes to lf* files and whether they can use your initializer instead. 6. Replaced hash with lock-free hash: test result changes. Is the order of rows stable? perhaps you should use --sort_results ? in MDL_lock constructor, I'd set m_strategy to 0 (ori to 1 or, may be, keep it uninitialized) as a safety measure, to ensure it's not used before MDL_lock::lf_hash_initializer(). Otherwise looks ok. Mostly a straightforward change. Regards, Sergei

2 3

[Maria-developers] passwordless mariadb root login with auht_socket in Debian
by Otto Kekäläinen 03 Mar '15

03 Mar '15

Hello Daniel! It seems there are some issues with the passwordless login patch you sent me some weeks ago (https://github.com/ottok/mariadb-10.0/pull/3) Can you help me debug the failing postinst script? I added 'set -x' to the mariadb-server-10.0.postinst script and when it is run during the configure phase it will exit with code 1 and make dpkg abort the installation. Last lines of execution: SET sql_log_bin=0; CREATE TABLE IF NOT EXISTS plugin (name char(64) COLLATE utf8_bin NOT NULL DEFAULT '\'''\'', dl char(128) COLLATE utf8_bin NOT NULL DEFAULT '\'''\'', PRIMARY KEY (name)) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='\''MySQL plugins'\'';' + logger -p daemon.err -t mysqld_safe -i + /usr/sbin/mysqld --bootstrap --user=mysql --skip-grant-tables --default-storage-engine=myisam --plugin-load-add=auth_socket + set -e + echo 'SET sql_log_bin=0; USE mysql; DELIMITER //; CREATE PROCEDURE debian_plugin_install(IN plugin_name CHAR(50), IN soname CHAR(50)) BEGIN DECLARE CONTINUE HANDLER FOR NOT FOUND EXECUTE inst_plug; set @plugin_name=plugin_name; set @soname=soname ;set @install_plugin=CONCAT("INSTALL PLUGIN ",@plugin_name," SONAME '\''", @soname, "'\''");PREPARE inst_plug FROM @install_plugin ; select PLUGIN_NAME INTO @a from information_schema.plugins where PLUGIN_NAME=@plugin_name AND PLUGIN_STATUS='\''ACTIVE'\'' AND PLUGIN_TYPE='\''AUTHENTICATION'\'' AND PLUGIN_LIBRARY LIKE concat(@soname,'\''%'\'' ); DEALLOCATE PREPARE inst_plug; END// CALL debian_plugin_install('\''unix_socket'\'', '\''auth_socket'\'') // DROP PROCEDURE debian_plugin_install//' + logger -p daemon.err -t mysqld_safe -i + /usr/sbin/mysqld --bootstrap --user=mysql --skip-grant-tables --default-storage-engine=myisam --plugin-load-add=auth_socket timo@debunstable:~$ echo $? 1 timo@debunstable:~$ echo $? 0 This implies that the exit code from the line '/usr/sbin/mysqld --bootstrap --user=mysql --skip-grant-tables --default-storage-engine=myisam --plugin-load-add=auth_socket' is 1. Any ideas why? If you want to test it yourself, add to you apt sources the test repo deb http://labs.seravo.fi/~otto/mariadb-repo/ mariabd-10.0-sid-amd64/

4 15

[Maria-developers] 10.0 is now on github
by Sergei Golubchik 02 Mar '15

02 Mar '15

Hi, Olivier, Kentoku. Since when we've moved 10.1 development to git (about a year ago), the plan always was to move other trees to git too. So, last month we've moved all our development to git, 5.5 and 10.0 trees are now on github, in the https://github.com/MariaDB/server repository. I still keep launchpad trees, lp:maria/10.0, in particular, to be able to merge 10.0-connect, 10.0-spider, and 10.0-mroonga. When needed, I merge these trees to lp:maria/10.0, and then merge from lp:maria/10.0 to a 10.0 branch in the git repository. But it would be great if you could also move the development to git and github. The easiest and safest approach would be as follows: 1. you tell me when to do it. 2. I do the last merge of your lp tree to 10.0 and then to git and push it to github. 3. you fork https://github.com/MariaDB/server under your github account and then you can create your branch and do anything you want there. 4. tell me the url of your forked tree and I'll configure buildbot to build it. this way you don't need to export your changes from bzr to git (which takes few hours) - I'll do it (it'll be a couple of minutes for me, because I've already run git-remote-bzr many times, and it has all the data cached). And the history will be safer, as we can be sure that all already exported commits won't be re-exported the second time. So, just tell me when you're ready :) By the way, if you haven't pushed anything since 10.0.17 - that is, there's nothing to merge - you skip the first two steps and fork https://github.com/MariaDB/server right away. Good luck! Regards, Sergei

1 0