- developers - lists.mariadb.org

Re: [Maria-developers] Rev 4384: MDEV-6916: Upgrade from MySQL to MariaDB breaks already created views
by Sergei Golubchik 17 Dec '14

17 Dec '14

Hi, Sanja! Ok, I've seen all four patches. Specific comments about the code are below and in other emails. But the main question is - why are you doing it, what problem does it solve. As you said, the use case is - a user upgrades *from MySQL* and *runs mysql_upgrade*. To cover this use case mysql_upgrade needs to 1) detect whether it's an upgrade from MySQL, and to 2) run a command to change the algorithm of all views. You've done a lot of changes that aren't related to that. If this is, indeed, the use case, than you only need to: 1. add MySQL detection in mysql_upgrade. You have it, is_mysql() function. 2. add the code to toggle the algorithm of all views. See? No command line options, no FORCE, no CHECK, nothing. Now, let's consider extensions to this minimal functionality. 3. mariadb-version in the view frm file. this is a good idea, makes views a lot more future-proof. ok. I don't see why other extensions might be needed. Now, how to toggle the algorithm. The most natural solution would be to use ALTER VIEW. Like ALTER VIEW v1 ALGORITHM=MERGE. Unfortunately, ALTER VIEW doesn't work that way, I don't know why it was implemented in that crazy way in MySQL, but it's mostly useless. CHECK should not change its arguments. REPAIR? May be. I'd still prefer ALTER, but I don't know whether it can be extended in 5.5 in a safe way. If not then REPAIR ... AFTER UPGRADE is probably the second best. Or even REPAIR VIEW ... AFTER UPGRADE. It won't conflict with anything, doesn't add new keywords. Drawback - we'll be stuck with that hack for quite a while :( Even better would be to make mysql_upgrade to scan the data directory and rewrite view frms. No hacks or new grammar in the server. No new features in 5.5 GA. Nothing to maintain for years. Almost perfect. But it requires mysql_upgrade to have write access to the datadir. It kind of needs it already, but it's optional. With this approach it'll be less optional than before. So, I don't know... The most reasonable and future-proof solution would be to use ALTER TABLE. But I don't know whether we'll be able to do it in 5.5, looks a bit risky. Then there are reasonably safe hacks like REPAIR VIEW. And most safe and quite clean mysql_upgrade only approach, that needs write access to the datadir. Opinions? Anyway, here are comments about the code, below: On Dec 11, sanja(a)askmonty.org wrote: > At file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6916/ > > ------------------------------------------------------------ > revno: 4384 > revision-id: sanja(a)askmonty.org-20141211180458-j8ajlibnvvydmj40 > parent: sanja(a)askmonty.org-20141207094825-tx163unk3mm4gkql > committer: sanja(a)askmonty.org > branch nick: work-maria-5.5-MDEV-6916 > timestamp: Thu 2014-12-11 19:04:58 +0100 > message: > MDEV-6916: Upgrade from MySQL to MariaDB breaks already created views > > mysql_upgrade now fix views from mysql. > > (+ fix new view detection) > === modified file 'client/mysql_upgrade.c' > --- a/client/mysql_upgrade.c 2014-03-27 21:26:58 +0000 > +++ b/client/mysql_upgrade.c 2014-12-11 18:04:58 +0000 > @@ -40,7 +40,8 @@ static char mysql_path[FN_REFLEN]; > static char mysqlcheck_path[FN_REFLEN]; > > static my_bool opt_force, opt_verbose, debug_info_flag, debug_check_flag, > - opt_systables_only, opt_version_check; > + opt_systables_only, opt_version_check, > + opt_mysql_upgrade= 0, opt_skip_mysql_upgrade= 0; > static my_bool opt_not_used, opt_silent; > static uint my_end_arg= 0; > static char *opt_user= (char*)"root"; > @@ -150,6 +151,14 @@ static struct my_option my_long_options[ > &opt_not_used, &opt_not_used, 0, GET_BOOL, NO_ARG, 1, 0, 0, 0, 0, 0}, > {"version", 'V', "Output version information and exit.", 0, 0, 0, > GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0}, > + {"mysql-upgrade", 'y', > + "Skip automatic detection MySQL and assume that we upgrade it", > + &opt_mysql_upgrade, &opt_mysql_upgrade, 0, GET_BOOL, NO_ARG, > + 0, 0, 0, 0, 0, 0}, > + {"skip-mysql-upgrade", 'Y', > + "Skip view algorithm upgrade from MySQL", > + &opt_skip_mysql_upgrade, &opt_skip_mysql_upgrade, 0, GET_BOOL, NO_ARG, > + 0, 0, 0, 0, 0, 0}, Those two are very bad option names. First, $ mysql_upgrade --mysql-upgrade looks plain weird. $ mysql_upgrade --skip-mysql-upgrade is even worse. And what about $ mysql_upgrade --mysql-upgrade --skip-mysql-upgrade Second, "skip" is the prefix automatically recognized by my_getopt. Normally, --skip-xxx is the same as --xxx=0. But now you're introducing --skip-mysql-upgrade which is very different from --skip-mysql-upgrade. While simultaneously adding support for --skip-skip-mysql-upgrade. A better option would be --fix-view-algorithm = { AUTO | ALWAYS | NEVER } > {"version-check", 'k', "Run this program only if its \'server version\' " > "matches the version of the server to which it's connecting, (enabled by " > "default); use --skip-version-check to avoid this check. Note: the \'server " > @@ -754,6 +771,23 @@ static int run_mysqlcheck_upgrade(void) > NULL); > } > > +static int run_mysqlcheck_views(void) > +{ > + if (!opt_mysql_upgrade) > + return 0; > + verbose("Phase 0: Fixing views"); No "Phase 0" please, renumber all phases to start from 1. > + print_conn_args("mysqlcheck"); > + return run_tool(mysqlcheck_path, > + NULL, /* Send output from mysqlcheck directly to screen */ > + "--no-defaults", > + ds_args.str, > + "--all-databases", > + "--view-upgrade", > + opt_verbose ? "--verbose": "", > + opt_silent ? "--silent": "", > + "2>&1", > + NULL); > +} > > === modified file 'client/mysqlcheck.c' > --- a/client/mysqlcheck.c 2014-02-17 10:00:51 +0000 > +++ b/client/mysqlcheck.c 2014-12-11 18:04:58 +0000 > @@ -42,7 +42,8 @@ static my_bool opt_alldbs = 0, opt_check > opt_medium_check = 0, opt_quick = 0, opt_all_in_1 = 0, > opt_silent = 0, opt_auto_repair = 0, ignore_errors = 0, > tty_password= 0, opt_frm= 0, debug_info_flag= 0, debug_check_flag= 0, > - opt_fix_table_names= 0, opt_fix_db_names= 0, opt_upgrade= 0; > + opt_fix_table_names= 0, opt_fix_db_names= 0, opt_upgrade= 0, > + opt_mariadb_upgrade= 0, opt_f_mariadb_upgrade= 0; > static my_bool opt_write_binlog= 1, opt_flush_tables= 0; > static uint verbose = 0, opt_mysql_port=0; > static int my_end_arg; > @@ -196,6 +197,12 @@ static struct my_option my_long_options[ > NO_ARG, 0, 0, 0, 0, 0, 0}, > {"version", 'V', "Output version information and exit.", 0, 0, 0, GET_NO_ARG, > NO_ARG, 0, 0, 0, 0, 0, 0}, > + {"view-upgrade", 'f', > + "Fix view algorithm view field if it is not new MariaDB view.", "Repair algorithm value for views created by MySQL (or MariaDB before 5.5.41)" > + 0, 0, 0, GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0}, > + {"force-view-upgrade", 'F', > + "Swap view algorithm view field.", "Always change view algorithm" > + 0, 0, 0, GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0}, > {0, 0, 0, 0, 0, 0, GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0} > }; Regards, Sergei

1 0

[Maria-developers] Invisible Indexes and .frm Change?
by James Briggs 16 Dec '14

16 Dec '14

Hi list. I'm looking into adding invisible indexes. There's 3 "scopes" I've thought of: 1) global and persistent (create table, alter table, create index) 2) global temporary (alter table t1 change index invisible global) 3) session temporary (alter table t1 change index invisible session) #1 likely means changing the .frm format, which would cause backwardcompatibility issues. Are there any upcoming changes to .frm planned? Any comments or suggestions? Thanks, James Briggs. -- Cassandra/MySQL DBA. Available in San Jose area or remote. cass_top: https://github.com/jamesbriggs/cassandra-top

2 1

Re: [Maria-developers] Rev 4350: MDEV-6985: MariaDB crashes on stored procedure call
by Sergei Golubchik 16 Dec '14

16 Dec '14

Hi, Sanja! On Nov 17, sanja(a)askmonty.org wrote: > ------------------------------------------------------------ > revno: 4350 > revision-id: sanja(a)askmonty.org-20141117132635-gf281ipiomifw1oc > parent: psergey(a)askmonty.org-20141113111559-cuhsyfhor141qo6a > committer: sanja(a)askmonty.org > branch nick: work-maria-5.5-MDEV-6985 > timestamp: Mon 2014-11-17 14:26:35 +0100 > message: > MDEV-6985: MariaDB crashes on stored procedure call > > Item_ident fixed to allow double cleanup(). > === modified file 'sql/item.cc' > --- a/sql/item.cc 2014-10-06 17:53:55 +0000 > +++ b/sql/item.cc 2014-11-17 13:26:35 +0000 > @@ -862,12 +862,20 @@ void Item_ident::cleanup() > field_name ? field_name : "(null)", > orig_field_name ? orig_field_name : "(null)")); > #endif Before pushing, please remove the above #ifdef, in a separate commit. thanks! > + bool was_fixed= fixed; > Item::cleanup(); > db_name= orig_db_name; > table_name= orig_table_name; > field_name= orig_field_name; > /* Store if this Item was depended */ > - can_be_depended= test(depended_from); > + if (was_fixed) > + { > + /* > + We can trust that depended_from set correctly only if this item > + was fixed > + */ > + can_be_depended= test(depended_from); > + } Are double cleanups generally normal? Why this item is cleaned up twice? > DBUG_VOID_RETURN; > } Regards, Sergei

2 2

Re: [Maria-developers] Rev 4382: Added creation time to the view frm
by Sergei Golubchik 16 Dec '14

16 Dec '14

Hi, Sanja! On Dec 03, sanja(a)askmonty.org wrote: > At file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6916/ > > ------------------------------------------------------------ > revno: 4382 > revision-id: sanja(a)askmonty.org-20141203234414-63zkpeq9eb0n9lc1 > parent: sanja(a)askmonty.org-20141203223759-316k8y35qdb4cgyq > committer: sanja(a)askmonty.org > branch nick: work-maria-5.5-MDEV-6916 > timestamp: Thu 2014-12-04 00:44:14 +0100 > message: > Added creation time to the view frm. > ALTER VIEW saves it. CREATE VIEW & CREATE OR REPLACE VIEW reset it. Please no. This has nothing to do with MDEV-6916, it's a separate task and should be reviewed and discussed separately, not sneaked in under the MDEV-6916 cover. Regards, Sergei

1 0

Re: [Maria-developers] Rev 4381: Adding mariadb-version on the view creation to view frm. (MDEV-6916 followup)
by Sergei Golubchik 16 Dec '14

16 Dec '14

Hi, Sanja! On Dec 03, sanja(a)askmonty.org wrote: > ------------------------------------------------------------ > revno: 4381 > revision-id: sanja(a)askmonty.org-20141203223759-316k8y35qdb4cgyq > parent: jplindst(a)mariadb.org-20141203113839-xkxdy39mq77vu99y > committer: sanja(a)askmonty.org > branch nick: work-maria-5.5-MDEV-6916 > timestamp: Wed 2014-12-03 23:37:59 +0100 > message: > Adding mariadb-version on the view creation to view frm. (MDEV-6916 followup) Looks ok, thanks Regards, Sergei

1 0

[Maria-developers] Review of MDEV-7112 Split HA_CREATE_INFO
by Sergei Golubchik 04 Dec '14

04 Dec '14

Hi. It think the split is good, I liked it. I didn't like that you've tried to put the refactoring of sql_db.cc and sql_lang.h in the same commit, please move it to a separate patch. MDEV-5359 should not depend on that. Anyway, see all my comments below: > diff --git a/sql/structs.h b/sql/structs.h > index 99561c5..b38685c 100644 > --- a/sql/structs.h > +++ b/sql/structs.h > @@ -471,4 +471,67 @@ class Discrete_intervals_list { > Discrete_interval* get_current() const { return current; }; > }; > > + > +struct DDL_options_st > +{ > +public: > + enum Options > + { > + OPT_NONE= 0, > + OPT_IF_NOT_EXISTS= 2, // CREATE TABLE IF NOT EXISTS > + OPT_LIKE= 4, // CREATE TABLE LIKE > + OPT_OR_REPLACE= 16, // CREATE OR REPLACE TABLE > + OPT_OR_REPLACE_SLAVE_GENERATED= 32,// REPLACE was added on slave, it was > + // not in the original query on master. > + OPT_IF_EXISTS= 64 > + }; > + > +private: > + Options m_options; > + > +protected: > + > +public: > + Options create_like_options() const > + { > + return (DDL_options_st::Options) > + (((uint) m_options) & (OPT_IF_NOT_EXISTS | OPT_OR_REPLACE)); > + } > + void init() { m_options= OPT_NONE; } > + void init(Options options) { m_options= options; } > + void set(Options other) > + { > + m_options= other; > + } > + void set(const DDL_options_st other) > + { > + m_options= other.m_options; > + } > + bool if_not_exists() const { return m_options & OPT_IF_NOT_EXISTS; } > + bool or_replace() const { return m_options & OPT_OR_REPLACE; } > + bool or_replace_slave_generated() const > + { return m_options & OPT_OR_REPLACE_SLAVE_GENERATED; } > + bool like() const { return m_options & OPT_LIKE; } > + bool if_exists() const { return m_options & OPT_IF_EXISTS; } > + void add(const DDL_options_st::Options other) > + { > + m_options= (Options) ((uint) m_options | (uint) other); > + } > + void add(const DDL_options_st &other) > + { > + add(other.m_options); > + } > + DDL_options_st operator|(const DDL_options_st &other) > + { > + add(other.m_options); > + return *this; > + } > + DDL_options_st operator|=(DDL_options_st::Options other) > + { > + add(other); > + return *this; > + } > +}; I don't particularly like this style of C++ing... It's way too verbose. but ok > + > + > #endif /* STRUCTS_INCLUDED */ > diff --git a/sql/handler.h b/sql/handler.h > index 0044556..7b6a5f4 100644 > --- a/sql/handler.h > +++ b/sql/handler.h > @@ -376,10 +376,7 @@ enum enum_alter_inplace_result { > #define HA_KEY_BLOB_LENGTH 2 > > #define HA_LEX_CREATE_TMP_TABLE 1 > -#define HA_LEX_CREATE_IF_NOT_EXISTS 2 > -#define HA_LEX_CREATE_TABLE_LIKE 4 > #define HA_CREATE_TMP_ALTER 8 > -#define HA_LEX_CREATE_REPLACE 16 One empty line here, please, HA_MAX_REC_LENGTH is not a flag > #define HA_MAX_REC_LENGTH 65535 > > /* Table caching type */ > @@ -1571,9 +1568,9 @@ enum enum_stats_auto_recalc { HA_STATS_AUTO_RECALC_DEFAULT= 0, > HA_STATS_AUTO_RECALC_ON, > HA_STATS_AUTO_RECALC_OFF }; > Could you add structure comments here? Like what you have in MDEV, "This will be passed to mysql_create_db, mysql_drop_db, mysqld_show_create_db", etc To explain what every structure is for, what kind of data it should store, and expected users (internal DB DDL, internal table DDL, handler, etc) > -struct HA_CREATE_INFO > +struct Table_contents_source_st > { > - CHARSET_INFO *table_charset, *default_table_charset; > + CHARSET_INFO *table_charset; > LEX_CUSTRING tabledef_version; > LEX_STRING connect_string; > const char *password, *tablespace; > @@ -1635,6 +1635,55 @@ struct HA_CREATE_INFO > }; > comments for all new structures, please > +struct Schema_specification_st > +{ > + CHARSET_INFO *default_table_charset; > + void init() > + { > + bzero(this, sizeof(*this)); > + } > +}; > + > + > +struct HA_CREATE_INFO: public Table_contents_source_st, > + public Schema_specification_st > +{ > + void init() > + { > + Table_contents_source_st::init(); > + Schema_specification_st::init(); > + } > +}; > + > + > +struct Table_specification_st: public HA_CREATE_INFO, > + public DDL_options_st > +{ > + // Deep initialization > + void init() > + { > + HA_CREATE_INFO::init(); > + DDL_options_st::init(); > + } > + void init(DDL_options_st::Options options) > + { > + HA_CREATE_INFO::init(); > + DDL_options_st::init(options); > + } > + /* > + Quick initialization, for parser. > + Most of the HA_CREATE_INFO is left uninitialized. > + It gets fully initialized in sql_yacc.yy, only when the parser > + scans a related keyword (e.g. CREATE, ALTER). > + */ > + void lex_start() > + { > + HA_CREATE_INFO::options= 0; > + DDL_options_st::init(); > + } > +}; > + > + > /** > In-place alter handler context. > > diff --git a/sql/sql_lang.h b/sql/sql_lang.h > new file mode 100644 > index 0000000..76a1340 > --- /dev/null > +++ b/sql/sql_lang.h > @@ -0,0 +1,79 @@ > +/* Copyright (c) 2014, MariaDB Foundation > + > + This program is free software; you can redistribute it and/or modify > + it under the terms of the GNU General Public License as published by > + the Free Software Foundation; version 2 of the License. > + > + This program is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + GNU General Public License for more details. > + > + You should have received a copy of the GNU General Public License > + along with this program; if not, write to the Free Software > + Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */ > + > +#ifndef SQL_LANG_INCLUDED > +#define SQL_LANG_INCLUDED > + > +namespace SQL { > + > +class Identifier: protected LEX_CSTRING > +{ > + void set(const LEX_STRING &other) > + { > + str= other.str; > + length= other.length; > + DBUG_ASSERT(str[length] == 0); > + } > +public: > + Identifier(const LEX_STRING &other) > + { > + set(other); > + } > + Identifier(const LEX *); > + const char *name() const { return str; } > + const size_t name_length() const { return length; } > +}; Eh? What all this has to do with MDEV-7112 "Split HA_CREATE_INFO" ? it's not mentioned there. If it's unrelated refactoring, please, put it in a separate patch. > + > + > +class Identifier_normalized: public Identifier > +{ > + char m_buf[NAME_LEN + 1 + MYSQL50_TABLE_NAME_PREFIX_LENGTH + 10]; > + void normalize(); > +public: > + Identifier_normalized(const LEX_STRING &other) > + :Identifier(other) { normalize(); } > + Identifier_normalized(const LEX *lex) > + :Identifier(lex) { normalize(); } > +}; > + > + > +namespace Statement { > + > + > +/* > + All SQL statements that can be executed directly. > + Statement instances are created using the data in thd->lex, > + previously populated by the SQL parser. > +*/ > +class Directly_executable > +{ > +public: > + /* > + Perform all stages of a direct SQL execution: > + - Check for validity (e.g. name in "CREATE SCHEMA name" is valid). > + - Check access > + - Apply filters (e.g. Rpl_filters on slave) > + - Perform WSREP_TO_ISOLATION > + - Execute the actual action > + */ > + virtual int exec_direct(THD *thd) const = 0; > +}; > + > +} // End of namescpace Statement > + > + > +} // End of namespace SQL > + > +#endif /* SQL_LANG_INCLUDED */ > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > index 0bbcca5..0edda65 100644 > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -1923,7 +1923,9 @@ bool MDL_deadlock_handler::handle_condition(THD *, > */ > > static bool > -open_table_get_mdl_lock(THD *thd, Open_table_context *ot_ctx, > +open_table_get_mdl_lock(THD *thd, > + const DDL_options_st &options, > + Open_table_context *ot_ctx, What's the point - you don't seem to be using options in this function? And that recursively means that you don't need 'options' in all other functions that only pass 'options' argument down to open_table_get_mdl_lock() but otherwise don't use it. > MDL_request *mdl_request, > uint flags, > MDL_ticket **mdl_ticket) > @@ -4176,8 +4182,8 @@ thr_lock_type read_lock_type_for_table(THD *thd, > DBUG_RETURN(FALSE); > > /* Check if CREATE TABLE without REPLACE was used */ > - create_table= (thd->lex->sql_command == SQLCOM_CREATE_TABLE && > - !(thd->lex->create_info.options & HA_LEX_CREATE_REPLACE)); > + create_table= thd->lex->sql_command == SQLCOM_CREATE_TABLE && > + !options.or_replace(); why, options are still in thd->lex, aren't they? > > if (!(flags & MYSQL_OPEN_SKIP_SCOPED_MDL_LOCK)) > { > diff --git a/sql/sql_db.h b/sql/sql_db.h > index 62d379c..dc62e6b 100644 > --- a/sql/sql_db.h > +++ b/sql/sql_db.h > @@ -35,13 +33,178 @@ bool mysql_opt_change_db(THD *thd, > bool my_dboptions_cache_init(void); > void my_dboptions_cache_free(void); > bool check_db_dir_existence(const char *db_name); > -bool load_db_opt(THD *thd, const char *path, HA_CREATE_INFO *create); > +bool load_db_opt(THD *thd, const char *path, Schema_specification_st *create); > bool load_db_opt_by_name(THD *thd, const char *db_name, > - HA_CREATE_INFO *db_create_info); > + Schema_specification_st *db_create_info); > CHARSET_INFO *get_default_db_collation(THD *thd, const char *db_name); > bool my_dbopt_init(void); > void my_dbopt_cleanup(void); > > + > +namespace SQL { > + > + > +namespace Statement { > + > + > +namespace Schema { > + > + > +// Schema name > +class Name: public Identifier_normalized > +{ > +public: > + Name(const LEX_STRING &other) :Identifier_normalized(other) { } > + Name(const LEX *lex) :Identifier_normalized(lex) { } > +#ifdef HAVE_REPLICATION > + bool check_if_slave_ignored(THD *thd) const; > +#endif > + bool check() const; > + bool check_with_error() const; > +}; > + > + > +// Schema definition options: IF EXISTS, IF NOT EXISTS > +class DDL_options: public DDL_options_st > +{ > +public: > + DDL_options() { init(); } > + DDL_options(Options options) > + { > + set(options); > + } > + DDL_options(const DDL_options_st options) > + { > + set(options); > + } > + DDL_options(const LEX *lex); > +}; > + > + > +// Schema specification, e.g. CHARACTER SET xxx > +class Specification: public Schema_specification_st > +{ > +public: > + Specification(const Schema_specification_st &info) > + :Schema_specification_st(info) > + { } > + Specification(const LEX *lex); > +}; > + > + > +/** > + A common class for all Schema DDL options: > + - CREATE SCHEMA > + - DROP SCHEMA > + - ALTER SCHEMA > + - ALTER SCHEMA UPGRADE DATA DIRECTORY NAME > +*/ > +class DDL: public Directly_executable > +{ > +protected: > + bool rm_db(THD *thd, const Name &name, bool if_exists, bool silent) const; > + > + int create_db(THD *thd, > + const Name &name, > + const DDL_options_st options, > + const Schema_specification_st &info, > + bool silent) const; > +}; > + > + > +class DDL_with_name: public DDL, public Name > +{ > + /* > + Execute a DDL statement. > + Assumes that Name has already been checked for validity. > + */ > + virtual bool exec(THD *thd) const = 0; > + virtual bool check_access(THD *) const = 0; > +protected: > + DDL_with_name(const Name &name): Name(name) { } > + DDL_with_name(const LEX *lex): Name(lex) { } > +public: > + int exec_direct(THD *thd) const; > +}; > + > + > +class DDL_with_name_and_options: public DDL_with_name, public DDL_options > +{ > +protected: > + DDL_with_name_and_options(const LEX *lex) > + :DDL_with_name(lex), DDL_options(lex) { } > + DDL_with_name_and_options(const Name &name) > + :DDL_with_name(name), DDL_options() { } > +}; > + > + > +class Create: public DDL_with_name_and_options, public Specification > +{ > + bool exec(THD *thd) const > + { > + return DDL::create_db(thd, *this, *this, *this, false); > + } > + bool check_access(THD *thd) const > + { > + return ::check_access(thd, CREATE_ACL, name(), NULL, NULL, 1, 0); > + } > +public: > + Create(const LEX *lex) > + :DDL_with_name_and_options(lex), Specification(lex) > + { } > + Create(const Name &name, const Schema_specification_st &info) > + :DDL_with_name_and_options(name), Specification(info) > + { } > +}; > + > + > +class Drop: public DDL_with_name_and_options > +{ > + bool exec(THD *thd) const > + { > + return rm_db(thd, *this, if_exists(), false); > + } > + bool check_access(THD *thd) const > + { > + return ::check_access(thd, DROP_ACL, name(), NULL, NULL, 1, 0); > + } > +public: > + Drop(const Name &name) :DDL_with_name_and_options(name) { } > + Drop(const LEX *lex) :DDL_with_name_and_options(lex) { } > +}; > + > + > +class Alter: public DDL_with_name, public Specification > +{ > + bool exec(THD *thd) const; > + bool check_access(THD *thd) const > + { > + return ::check_access(thd, ALTER_ACL, name(), NULL, NULL, 1, 0); > + } > +public: > + Alter(const LEX *lex) :DDL_with_name(lex), Specification(lex) { } > +}; > + > + > +class Upgrade: public DDL_with_name > +{ > + bool exec(THD *thd) const; > + bool check_access(THD *thd) const > + { > + return ::check_access(thd, ALTER_ACL, name(), NULL, NULL, 1, 0) || > + ::check_access(thd, DROP_ACL, name(), NULL, NULL, 1, 0) || > + ::check_access(thd, CREATE_ACL, name(), NULL, NULL, 1, 0); > + } > +public: > + Upgrade(const LEX *lex) :DDL_with_name(lex) { } > + int exec_direct(THD *thd) const; > +}; > + > + > +} // End of namespace SQL::Statement::Schema > +} // End of namespace SQL::Statement > +} // End of namespace SQL Please, no. This and sql_lang.h - make it a separate patch This one should be *only* about splitting HA_CREATE_INFO > + > #define MY_DB_OPT_FILE "db.opt" > > #endif /* SQL_DB_INCLUDED */ > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index b905d8b..c59332e 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -4181,143 +4176,21 @@ static bool do_execute_sp(THD *thd, sp_head *sp) > } > break; > case SQLCOM_CREATE_DB: > - { > - /* > - As mysql_create_db() may modify HA_CREATE_INFO structure passed to > - it, we need to use a copy of LEX::create_info to make execution > - prepared statement- safe. > - */ > - HA_CREATE_INFO create_info(lex->create_info); > - if (check_db_name(&lex->name)) > - { > - my_error(ER_WRONG_DB_NAME, MYF(0), lex->name.str); > - break; > - } > - /* > - If in a slave thread : > - CREATE DATABASE DB was certainly not preceded by USE DB. > - For that reason, db_ok() in sql/slave.cc did not check the > - do_db/ignore_db. And as this query involves no tables, tables_ok() > - above was not called. So we have to check rules again here. > - */ > -#ifdef HAVE_REPLICATION > - if (thd->slave_thread) > - { > - rpl_filter= thd->system_thread_info.rpl_sql_info->rpl_filter; > - if (!rpl_filter->db_ok(lex->name.str) || > - !rpl_filter->db_ok_with_wild_table(lex->name.str)) > - { > - my_message(ER_SLAVE_IGNORED_TABLE, ER(ER_SLAVE_IGNORED_TABLE), MYF(0)); > - break; > - } > - } > -#endif > - if (check_access(thd, CREATE_ACL, lex->name.str, NULL, NULL, 1, 0)) > - break; > - WSREP_TO_ISOLATION_BEGIN(lex->name.str, NULL, NULL) > - res= mysql_create_db(thd, lex->name.str, &create_info, 0); separate refactoring patch, please > + if ((res= SQL::Statement::Schema::Create(lex).exec_direct(thd)) < 0) > + goto error; > break; > - } > case SQLCOM_DROP_DB: > - { > - if (check_db_name(&lex->name)) > - { > - my_error(ER_WRONG_DB_NAME, MYF(0), lex->name.str); > - break; > - } > - /* > - If in a slave thread : > - DROP DATABASE DB may not be preceded by USE DB. > - For that reason, maybe db_ok() in sql/slave.cc did not check the > - do_db/ignore_db. And as this query involves no tables, tables_ok() > - above was not called. So we have to check rules again here. > - */ > -#ifdef HAVE_REPLICATION > - if (thd->slave_thread) > - { > - rpl_filter= thd->system_thread_info.rpl_sql_info->rpl_filter; > - if (!rpl_filter->db_ok(lex->name.str) || > - !rpl_filter->db_ok_with_wild_table(lex->name.str)) > - { > - my_message(ER_SLAVE_IGNORED_TABLE, ER(ER_SLAVE_IGNORED_TABLE), MYF(0)); > - break; > - } > - } > -#endif > - if (check_access(thd, DROP_ACL, lex->name.str, NULL, NULL, 1, 0)) > - break; > - WSREP_TO_ISOLATION_BEGIN(lex->name.str, NULL, NULL) > - res= mysql_rm_db(thd, lex->name.str, lex->check_exists, 0); > + if ((res= SQL::Statement::Schema::Drop(lex).exec_direct(thd)) < 0) > + goto error; > break; > - } > case SQLCOM_ALTER_DB_UPGRADE: > - { > - LEX_STRING *db= & lex->name; > -#ifdef HAVE_REPLICATION > - if (thd->slave_thread) > - { > - rpl_filter= thd->system_thread_info.rpl_sql_info->rpl_filter; > - if (!rpl_filter->db_ok(db->str) || > - !rpl_filter->db_ok_with_wild_table(db->str)) > - { > - res= 1; > - my_message(ER_SLAVE_IGNORED_TABLE, ER(ER_SLAVE_IGNORED_TABLE), MYF(0)); > - break; > - } > - } > -#endif > - if (check_db_name(db)) > - { > - my_error(ER_WRONG_DB_NAME, MYF(0), db->str); > - break; > - } > - if (check_access(thd, ALTER_ACL, db->str, NULL, NULL, 1, 0) || > - check_access(thd, DROP_ACL, db->str, NULL, NULL, 1, 0) || > - check_access(thd, CREATE_ACL, db->str, NULL, NULL, 1, 0)) > - { > - res= 1; > - break; > - } > - WSREP_TO_ISOLATION_BEGIN(db->str, NULL, NULL) > - res= mysql_upgrade_db(thd, db); > - if (!res) > - my_ok(thd); > + if ((res= SQL::Statement::Schema::Upgrade(lex).exec_direct(thd)) < 0) > + goto error; > break; > - } > case SQLCOM_ALTER_DB: > - { > - LEX_STRING *db= &lex->name; > - HA_CREATE_INFO create_info(lex->create_info); > - if (check_db_name(db)) > - { > - my_error(ER_WRONG_DB_NAME, MYF(0), db->str); > - break; > - } > - /* > - If in a slave thread : > - ALTER DATABASE DB may not be preceded by USE DB. > - For that reason, maybe db_ok() in sql/slave.cc did not check the > - do_db/ignore_db. And as this query involves no tables, tables_ok() > - above was not called. So we have to check rules again here. > - */ > -#ifdef HAVE_REPLICATION > - if (thd->slave_thread) > - { > - rpl_filter= thd->system_thread_info.rpl_sql_info->rpl_filter; > - if (!rpl_filter->db_ok(db->str) || > - !rpl_filter->db_ok_with_wild_table(db->str)) > - { > - my_message(ER_SLAVE_IGNORED_TABLE, ER(ER_SLAVE_IGNORED_TABLE), MYF(0)); > - break; > - } > - } > -#endif > - if (check_access(thd, ALTER_ACL, db->str, NULL, NULL, 1, 0)) > - break; > - WSREP_TO_ISOLATION_BEGIN(db->str, NULL, NULL) > - res= mysql_alter_db(thd, db->str, &create_info); > + if ((res= SQL::Statement::Schema::Alter(lex).exec_direct(thd)) < 0) > + goto error; > break; > - } > case SQLCOM_SHOW_CREATE_DB: > { > char db_name_buff[NAME_LEN+1]; > diff --git a/sql/sql_show.cc b/sql/sql_show.cc > index 0541cbc..2536255 100644 > --- a/sql/sql_show.cc > +++ b/sql/sql_show.cc > @@ -1696,14 +1695,14 @@ int show_create_table(THD *thd, TABLE_LIST *table_list, String *packet, > > packet->append(STRING_WITH_LEN("CREATE ")); > if (create_info_arg && > - (create_info_arg->org_options & HA_LEX_CREATE_REPLACE || > + ((create_info_arg->or_replace() && > + !create_info_arg->or_replace_slave_generated()) || that's a bit strange. Could you explain this? why org_options & HA_LEX_CREATE_REPLACE is translated to or_replace() && !or_replace_slave_generated() > create_info_arg->table_was_deleted)) > packet->append(STRING_WITH_LEN("OR REPLACE ")); > if (share->tmp_table) > packet->append(STRING_WITH_LEN("TEMPORARY ")); > packet->append(STRING_WITH_LEN("TABLE ")); > - if (create_info_arg && > - (create_info_arg->options & HA_LEX_CREATE_IF_NOT_EXISTS)) > + if (create_info_arg && create_info_arg->if_not_exists()) > packet->append(STRING_WITH_LEN("IF NOT EXISTS ")); > if (table_list->schema_table) > alias= table_list->schema_table->table_name; > diff --git a/sql/sql_table.cc b/sql/sql_table.cc > index b991215..7bb293b 100644 > --- a/sql/sql_table.cc > +++ b/sql/sql_table.cc > @@ -4628,7 +4628,8 @@ int create_table_impl(THD *thd, > const char *orig_db, const char *orig_table_name, > const char *db, const char *table_name, > const char *path, > - HA_CREATE_INFO *create_info, > + const DDL_options_st options, > + HA_CREATE_INFO &create_info, why? I generally prefer pointers over C++ references - one can immediately see what variables can be modified by the function. is there a compelling reason to use references here? > Alter_info *alter_info, > int create_table_mode, > bool *is_trans, > @@ -5273,9 +5275,10 @@ bool mysql_create_like_table(THD* thd, TABLE_LIST* table, > */ > > /* Copy temporarily the statement flags to thd for lock_table_names() */ > + // QQ: is this really needed??? may be not anymore, as you pass flags separately > uint save_thd_create_info_options= thd->lex->create_info.options; > - thd->lex->create_info.options|= create_info->options; > - res= open_tables(thd, &thd->lex->query_tables, &not_used, 0); > + thd->lex->create_info.options|= create_info.options; > + res= open_tables(thd, create_info, &thd->lex->query_tables, &not_used, 0); > thd->lex->create_info.options= save_thd_create_info_options; > > if (res) > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index 632bdd2..0f2c0e0 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -1621,7 +1623,8 @@ bool my_yyoverflow(short **a, YYSTYPE **b, ulong *yystacksize); > IDENT_sys TEXT_STRING_sys TEXT_STRING_literal > NCHAR_STRING opt_component key_cache_name > sp_opt_label BIN_NUM label_ident TEXT_STRING_filesystem ident_or_empty > - opt_constraint constraint opt_ident opt_if_not_exists_ident > + opt_constraint constraint opt_ident > + opt_if_not_exists_opt_table_element_name can you please try to come up with shorter identifier names? 40 characters - that's a bit too long :) > > %type <lex_str_ptr> > opt_table_alias > diff --git a/storage/xtradb/handler/ha_innodb.cc b/storage/xtradb/handler/ha_innodb.cc > index f3b1167..6b1942e 100644 > --- a/storage/xtradb/handler/ha_innodb.cc > +++ b/storage/xtradb/handler/ha_innodb.cc > @@ -11349,7 +11349,7 @@ static __attribute__((nonnull, warn_unused_result)) > > /* Do not use DATA DIRECTORY with TEMPORARY TABLE. */ > if (create_info->data_file_name > - && create_info->options & HA_LEX_CREATE_TMP_TABLE) { > + && create_info->tmp_table()) { nope, you should not change storage engine API here. tmp_table() was a convenience wrapper, old way should still work > push_warning( > thd, Sql_condition::WARN_LEVEL_WARN, > ER_ILLEGAL_HA_CREATE_OPTION, Regards, Sergei

2 4

[Maria-developers] better execution plan possible with INNER JOIN of mroonga (or whatever storage engine) ?
by Kazuhiko Shiozaki 03 Dec '14

03 Dec '14

Hello ! I am investigating a slow query and have a question about execution plan with mroonga (possibly any storage engine?). (with MariaDB 10.0.15.) Here are tables : CREATE TABLE catalog ( uid bigint(20) unsigned NOT NULL, path varchar(255) COLLATE utf8_unicode_ci NOT NULL DEFAULT '', PRIMARY KEY (uid) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci; CREATE TABLE catalog_full_text ( uid bigint(20) unsigned NOT NULL, title varchar(255) COLLATE utf8_unicode_ci DEFAULT '', PRIMARY KEY (uid), FULLTEXT KEY title (title) COMMENT 'parser "TokenBigramSplitSymbolAlphaDigit"' ) ENGINE=Mroonga DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci; CREATE TABLE category ( uid bigint(20) unsigned NOT NULL, category_uid bigint(20) unsigned NOT NULL DEFAULT '0', base_category_uid bigint(20) unsigned NOT NULL DEFAULT '0', PRIMARY KEY (uid,category_uid,base_category_uid) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci; Here is the query : SELECT DISTINCT catalog.path, catalog.uid FROM ( ( catalog AS catalog INNER JOIN ( category AS category INNER JOIN catalog_full_text AS source_title_full_text ON category.base_category_uid = 18482 AND source_title_full_text.uid = category.category_uid ) ON category.uid = catalog.uid ) INNER JOIN catalog_full_text AS title_full_text ON title_full_text.uid = catalog.uid ) WHERE MATCH (source_title_full_text.title) AGAINST ('bar' IN BOOLEAN MODE) AND MATCH (title_full_text.title) AGAINST ('foo' IN BOOLEAN MODE) Here is the output of EXPLAIN : *************************** 1. row *************************** id: 1 select_type: SIMPLE table: source_title_full_text type: fulltext possible_keys: PRIMARY,title key: title key_len: 0 ref: rows: 1 Extra: Using where with pushed condition; Using temporary *************************** 2. row *************************** id: 1 select_type: SIMPLE table: title_full_text type: fulltext possible_keys: PRIMARY,title key: title key_len: 0 ref: rows: 1 Extra: Using where with pushed condition *************************** 3. row *************************** id: 1 select_type: SIMPLE table: catalog type: eq_ref possible_keys: PRIMARY key: PRIMARY key_len: 8 ref: test.title_full_text.uid rows: 1 Extra: *************************** 4. row *************************** id: 1 select_type: SIMPLE table: category type: eq_ref possible_keys: PRIMARY key: PRIMARY key_len: 24 ref: test.title_full_text.uid,test.source_title_full_text.uid,const rows: 1 Extra: Using index; Distinct As you see, we have two full-text queries at the beginning and both have 'rows = 1'. In a slow query case, 25k records mathces with the first full-text query and 1.6k records matches with the second full-text query. When I manually modify the order of INNER JOIN so that the 'lighter' full-text query appears first, EXPLAIN output is different as well and the performance is much improved. Accoding to Kohei, a Mroonga developer, we always have 'rows = 1' for mroonga full-text query because MariaDB does not request such value to mroonga (JOIN::get_examined_rows() in sql/sql_select.cc), and it is possible to modify mroonga to return an estimation of rows if MariaDB has such an API. So here are my questions : * can MariaDB request an 'estimation of rows' to mroonga (or whatever storage engine) ? * can such information help to decide a better execution plan with INNER JOIN like above ? Thanks in advance ! -- Kazuhiko Shiozaki, Nexedi SA Senior Consultant Nexedi: Consulting and Development of Free / Open Source Software http://www.nexedi.com/ ERP5: Full Featured High End Open Source ERP http://www.erp5.com/

3 3

[Maria-developers] Google's patches to MariaDB are available in the opensource repo
by Pavel Ivanov 03 Dec '14

03 Dec '14

Hi all, Effective immediately the full opensource tree of changes made to MariaDB by Google to make it work for our internal applications is available at https://code.google.com/p/google-mysql/. Note: I didn't update the main page on the site to reflect the changes yet, but the sources are there and the master branch points to the tip of MariaDB tree. Note #2: this tree is not an exact copy of our internal tree, and it's not intended to reflect verbatim our internal development process. It's just a set of changes split roughly one feature per commit with a grouping of several commits related to the same feature set into branches. That means that whenever we find a bug in some of our commits we'll squash the fix in the appropriate place rebuilding the whole git tree on top of that. As a consequence of that links to specific commits will not last forever, though links to specific branches likely will. Note #3: unfortunately code.google.com doesn't have a good visualization of branches in the git tree, so it's not easy to see which commits belong to which branch in our structure of branches stacked on top of each other. So the easiest way to see that is to clone the tree on your local disk and run gitk on it (or other tool that you find more convenient). Besides raising a general awareness of existence of our opensource tree, I'd like to draw attention of MariaDB developers especially to the following branches. I think many commits from there can be considered for inclusion into MariaDB proper. 16-encryption https://code.google.com/p/google-mysql/source/list?name=mariadb-10.0.12%2F1… Encryption of different subsystems writing to disk (not all subsystems included yet, but will be soon). 09-rpl https://code.google.com/p/google-mysql/source/list?name=mariadb-10.0.12%2F0… Replication-related fixes and improvements. 01-base https://code.google.com/p/google-mysql/source/list?name=mariadb-10.0.12%2F0… Mostly basic compilation and testing fixes. 03-privileges https://code.google.com/p/google-mysql/source/list?name=mariadb-10.0.12%2F0… Fixes for some quirks in privilege subsystem (aka sql_acl.cc). If you want to discuss any changes that you see in the tree feel free to reply to this email or start a separate email thread. Thank you, Pavel

2 1

[Maria-developers] Some patches for the C client library
by Daniel Schepler 02 Dec '14

02 Dec '14

While I was creating Debian packages for the "MariaDB Client Library for C" I came across several minor issues that I corrected. I thought I'd post the patches here in case you might be interested in integrating them; I'd have no problem with contributing them under the MCA if that's necessary (i.e. if they're even big enough to require such a thing). Each one has a fairly complete description of its purpose at the start. I've also noticed that mariadb_config --libs prints out something like ... -l-lz if WITH_EXTERNAL_ZLIB=ON. However, for the Debian packages I instead patched mariadb_config to remove the indirect dependencies altogether, as we generally do, so I didn't write a fix for the -l-lz issue. Also, it seems that if I try to build with WITH_SQLITE=ON then I get a build error, so I haven't enabled that in the Debian packages. -- Daniel Schepler

2 2

[Maria-developers] crash, reporting bugs
by Roberto Spadim 02 Dec '14

02 Dec '14

hi guys, i have a consistent crash (mdev-7182) but this email is not about it... when i got the crash, i got this info: ... To report this bug, see http://kb.askmonty.org/en/reporting-bugs ... should we update this url to the current one? https://mariadb.com/kb/en/mariadb/development/debugging-mariadb/reporting-b… thanks -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

3 3

Re: [Maria-developers] [Commits] 020e569: MDEV-4412 : SLOW QUERY LOG - add affected rows (UPDATE / DELETE) in slow query log
by Sergei Golubchik 30 Nov '14

30 Nov '14

Hi, Nirbhay! On Nov 29, nirbhay(a)mariadb.com wrote: > revision-id: 020e569dc7f638b6add8a043b1c9ff308f688ffa > parent(s): 7b55b67de5fdfe91283357fe6c7ccc3f9e355925 > committer: Nirbhay Choubey > branch nick: b4412 > timestamp: 2014-11-29 19:05:32 -0500 > message: > > MDEV-4412 : SLOW QUERY LOG - add affected rows (UPDATE / DELETE) in slow query log > > Added Rows_affected to slow query log & mysql.slow_log table. > > diff --git a/scripts/mysql_system_tables_fix.sql b/scripts/mysql_system_tables_fix.sql > index b9bb59c..aeaf919 100644 > --- a/scripts/mysql_system_tables_fix.sql > +++ b/scripts/mysql_system_tables_fix.sql > @@ -249,7 +249,8 @@ SET GLOBAL general_log = @old_log_state; > SET @old_log_state = @@global.slow_query_log; > SET GLOBAL slow_query_log = 'OFF'; > ALTER TABLE slow_log > - ADD COLUMN thread_id BIGINT(21) UNSIGNED NOT NULL AFTER sql_text; > + ADD COLUMN thread_id BIGINT(21) UNSIGNED NOT NULL AFTER sql_text, > + ADD COLUMN rows_affected INTEGER NOT NULL AFTER thread_id; > ALTER TABLE slow_log > MODIFY start_time TIMESTAMP(6) NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, > MODIFY user_host MEDIUMTEXT NOT NULL, I am not sure it'll work. It would work if you need to upgrade a table that has neither thread_id nor rows_affected column to the latest version. So that ALTER would've been an acceptable solution if both thread_id and rows_affected were added in the same MariaDB version. But they were not and there are quite a few MariaDB releases where slow_log table has thread_id and doesn't have rows_affected. I suspect in that case ALTER will fail. But if you'd like you can try a small test case and see whether such an ALTER will add a second column even if the first one already exists. Better add rows_affected in a separate ALTER TABLE statement. Regards, Sergei

1 0

[Maria-developers] Patch: CREATE [[NO] FORCE] VIEW Options
by James Briggs 30 Nov '14

30 Nov '14

Hi List. Below is my patch that implements the CREATE [[NO] FORCE] VIEW options against MariaDB 10.1.0. Notes: - It's a fairly solid patch, but I think the best thing is to commit it to alpha and let it bake for a while. - it passes t/view.test: # ./mysql-test-run.pl view Logging: ./mysql-test-run.pl view vardir: /usr/local/mariadb-10.1.0/mysql-test/var MariaDB Version 10.1.0-MariaDB-debug TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- main.view [ pass ] 1896 -------------------------------------------------------------------------- The servers were restarted 0 times Spent 1.896 of 7 seconds executing testcases Completed: All 1 tests were successful. - I wrote tests/view.pl which does 8,000 tests. It passes. (I will include it in a separate email.) - Please take a look at the style, and the 411 comments. Thanks, James Briggs. -- Available for MySQL/NoSQL DBA/Programming in San Jose area or remote. $ cat create_force_view.patch --- ../mariadb-10.1.0/sql/sql_view.h 2014-06-27 04:50:36.000000000 -0700 +++ sql/sql_view.h 2014-09-02 02:35:42.000000000 -0700 @@ -29,10 +29,10 @@ /* Function declarations */ bool create_view_precheck(THD *thd, TABLE_LIST *tables, TABLE_LIST *view, - enum_view_create_mode mode); + enum_view_create_mode mode, enum_view_create_force force); bool mysql_create_view(THD *thd, TABLE_LIST *view, - enum_view_create_mode mode); + enum_view_create_mode mode, enum_view_create_force force); bool mysql_make_view(THD *thd, File_parser *parser, TABLE_LIST *table, uint flags); --- ../mariadb-10.1.0/sql/sql_lex.h 2014-06-27 04:50:33.000000000 -0700 +++ sql/sql_lex.h 2014-09-02 01:21:10.000000000 -0700 @@ -170,6 +170,12 @@ VIEW_CREATE_OR_REPLACE // check only that there are not such table }; +enum enum_view_create_force +{ + VIEW_CREATE_NO_FORCE, // default - check that there are not such VIEW/table + VIEW_CREATE_FORCE, // check that there are not such VIEW/table, then ignore table object dependencies +}; + enum enum_drop_mode { DROP_DEFAULT, // mode is not specified @@ -2442,6 +2448,7 @@ }; enum enum_var_type option_type; enum enum_view_create_mode create_view_mode; + enum enum_view_create_force create_view_force; enum enum_drop_mode drop_mode; uint profile_query_id; --- ../mariadb-10.1.0/sql/sql_parse.cc 2014-06-27 04:50:34.000000000 -0700 +++ sql/sql_parse.cc 2014-09-02 02:34:31.000000000 -0700 @@ -4943,7 +4943,7 @@ Note: SQLCOM_CREATE_VIEW also handles 'ALTER VIEW' commands as specified through the thd->lex->create_view_mode flag. */ - res= mysql_create_view(thd, first_table, thd->lex->create_view_mode); + res= mysql_create_view(thd, first_table, thd->lex->create_view_mode, thd->lex->create_view_force); break; } case SQLCOM_DROP_VIEW: --- ../mariadb-10.1.0/sql/sql_yacc.yy 2014-06-27 04:50:37.000000000 -0700 +++ sql/sql_yacc.yy 2014-09-05 17:19:29.000000000 -0700 @@ -1851,7 +1851,7 @@ statement sp_suid sp_c_chistics sp_a_chistics sp_chistic sp_c_chistic xa opt_field_or_var_spec fields_or_vars opt_load_data_set_spec - view_algorithm view_or_trigger_or_sp_or_event + view_algorithm view_or_trigger_or_sp_or_event view_force_option definer_tail no_definer_tail view_suid view_tail view_list_opt view_list view_select view_check_option trigger_tail sp_tail sf_tail udf_tail event_tail @@ -2446,6 +2446,7 @@ VIEW_CREATE_OR_REPLACE); Lex->create_view_algorithm= DTYPE_ALGORITHM_UNDEFINED; Lex->create_view_suid= TRUE; + Lex->create_view_force= VIEW_CREATE_NO_FORCE; /* initialize just in case */ } view_or_trigger_or_sp_or_event { @@ -15887,6 +15888,15 @@ | event_tail ; +view_force_option: + /* empty */ /* 411 - is there a cleaner way of initializing here? */ + { Lex->create_view_force = VIEW_CREATE_NO_FORCE; } + | NO_SYM FORCE_SYM + { Lex->create_view_force = VIEW_CREATE_NO_FORCE; } + | FORCE_SYM + { Lex->create_view_force = VIEW_CREATE_FORCE; } + ; + /************************************************************************** DEFINER clause support. @@ -15944,7 +15954,7 @@ ; view_tail: - view_suid VIEW_SYM table_ident + view_suid view_force_option VIEW_SYM table_ident { LEX *lex= thd->lex; lex->sql_command= SQLCOM_CREATE_VIEW; --- ../mariadb-10.1.0/sql/sql_view.cc 2014-06-27 04:50:36.000000000 -0700 +++ sql/sql_view.cc 2014-09-05 19:33:58.000000000 -0700 @@ -248,7 +248,7 @@ */ bool create_view_precheck(THD *thd, TABLE_LIST *tables, TABLE_LIST *view, - enum_view_create_mode mode) + enum_view_create_mode mode, enum_view_create_force force) { LEX *lex= thd->lex; /* first table in list is target VIEW name => cut off it */ @@ -259,7 +259,7 @@ DBUG_ENTER("create_view_precheck"); /* - Privilege check for view creation: + Privilege check for view creation with default (NO FORCE): - user has CREATE VIEW privilege on view table - user has DROP privilege in case of ALTER VIEW or CREATE OR REPLACE VIEW @@ -272,6 +272,7 @@ checked that we have not more privileges on correspondent column of view table (i.e. user will not get some privileges by view creation) */ + if ((check_access(thd, CREATE_VIEW_ACL, view->db, &view->grant.privilege, &view->grant.m_internal, @@ -285,6 +286,11 @@ check_grant(thd, DROP_ACL, view, FALSE, 1, FALSE)))) goto err; + if (force) { + res = false; + DBUG_RETURN(res || thd->is_error()); + } + for (sl= select_lex; sl; sl= sl->next_select()) { for (tbl= sl->get_table_list(); tbl; tbl= tbl->next_local) @@ -369,7 +375,7 @@ #else bool create_view_precheck(THD *thd, TABLE_LIST *tables, TABLE_LIST *view, - enum_view_create_mode mode) + enum_view_create_mode mode, enum_view_create_force force) { return FALSE; } @@ -391,7 +397,7 @@ */ bool mysql_create_view(THD *thd, TABLE_LIST *views, - enum_view_create_mode mode) + enum_view_create_mode mode, enum_view_create_force force) { LEX *lex= thd->lex; bool link_to_local; @@ -425,14 +431,13 @@ goto err; } - if ((res= create_view_precheck(thd, tables, view, mode))) + if (res= create_view_precheck(thd, tables, view, mode, force)) goto err; lex->link_first_table_back(view, link_to_local); view->open_type= OT_BASE_ONLY; - if (open_temporary_tables(thd, lex->query_tables) || - open_and_lock_tables(thd, lex->query_tables, TRUE, 0)) + if (open_temporary_tables(thd, lex->query_tables) || (!force && open_and_lock_tables(thd, lex->query_tables, TRUE, 0))) { view= lex->unlink_first_table(&link_to_local); res= TRUE; @@ -513,6 +518,7 @@ } } +if (!force) { /* prepare select to resolve all fields */ lex->context_analysis_only|= CONTEXT_ANALYSIS_ONLY_VIEW; if (unit->prepare(thd, 0, 0)) @@ -612,6 +618,7 @@ } } #endif +} res= mysql_register_view(thd, view, mode); @@ -621,7 +628,7 @@ meta-data changes after ALTER VIEW. */ - if (!res) + // if (!res) + if (!res && !force) /* 411 - solves segfault problems with CREATE FORCE VIEW option sometimes */ tdc_remove_table(thd, TDC_RT_REMOVE_ALL, view->db, view->table_name, false); if (mysql_bin_log.is_open()) @@ -908,6 +915,8 @@ fn_format(path_buff, file.str, dir.str, "", MY_UNPACK_FILENAME); path.length= strlen(path_buff); if (ha_table_exists(thd, view->db, view->table_name, NULL)) { if (mode == VIEW_CREATE_NEW) --- ../mariadb-10.1.0/mysql-test/t/view.test 2014-06-27 04:50:30.000000000 -0700 +++ mysql-test/t/view.test 2014-09-06 00:23:32.000000000 -0700 @@ -5263,4 +5263,17 @@ --echo # ----------------------------------------------------------------- --echo # -- End of 10.0 tests. --echo # ----------------------------------------------------------------- + +create no force view v1 as select 1; +drop view if exists v1; + +create force view v1 as select 1; +drop view if exists v1; + +create force view v1 as select * from missing_base_table; +drop view if exists v1; + +--echo # ----------------------------------------------------------------- +--echo # -- End of 10.1 tests. +--echo # ----------------------------------------------------------------- SET optimizer_switch=@save_optimizer_switch;

2 2

Re: [Maria-developers] MDEV-4412 : SLOW QUERY LOG - add affected rows (UPDATE / DELETE) in slow query log
by Sergei Golubchik 30 Nov '14

30 Nov '14

Hi, Nirbhay! On Nov 28, nirbhay(a)mariadb.com wrote: > revision-id: 7a5559421f4f3dda44d04bf946e79661d09b4dc2 > parent(s): 7b55b67de5fdfe91283357fe6c7ccc3f9e355925 > committer: Nirbhay Choubey > branch nick: b4412 > timestamp: 2014-11-28 20:53:02 -0500 > message: > > MDEV-4412 : SLOW QUERY LOG - add affected rows (UPDATE / DELETE) in slow query log > > Added Rows_affected to slow query log & mysql.slow_log table. Just a couple of comments: > diff --git a/scripts/mysql_system_tables_fix.sql b/scripts/mysql_system_tables_fix.sql > index b9bb59c..4973dc9 100644 > --- a/scripts/mysql_system_tables_fix.sql > +++ b/scripts/mysql_system_tables_fix.sql > @@ -262,7 +262,8 @@ ALTER TABLE slow_log > MODIFY insert_id INTEGER NOT NULL, > MODIFY server_id INTEGER UNSIGNED NOT NULL, > MODIFY sql_text MEDIUMTEXT NOT NULL, > - MODIFY thread_id BIGINT(21) UNSIGNED NOT NULL; > + MODIFY thread_id BIGINT(21) UNSIGNED NOT NULL, > + MODIFY rows_affected INTEGER NOT NULL; Eh? That makes no sense. This file is for mysql_upgrade. Your ALTER TABLE would help if there was some MariaDB version where rows_affected column would've been nullable and you wanted to change it to NOT NULL. But there was no such MariaDB version. There was MariaDB version without rows_affected column at all. So, instead of this MODIFY rows_affected, you need to put another ALTER TABLE statement that adds rows_affected INTEGER NOT NULL column. > SET GLOBAL slow_query_log = @old_log_state; > > ALTER TABLE plugin > diff --git a/sql/log.cc b/sql/log.cc > index ff04105..24e5283 100644 > --- a/sql/log.cc > +++ b/sql/log.cc > @@ -2918,12 +2924,16 @@ bool MYSQL_QUERY_LOG::write(THD *thd, time_t current_time, > sprintf(lock_time_buff, "%.6f", ulonglong2double(lock_utime)/1000000.0); > if (my_b_printf(&log_file, > "# Thread_id: %lu Schema: %s QC_hit: %s\n" \ > - "# Query_time: %s Lock_time: %s Rows_sent: %lu Rows_examined: %lu\n", > + "# Query_time: %s Lock_time: %s Rows_sent: %lu Rows_examined: %lu\n" \ > + "# Rows_affected: %lu\n", I'd rather have Rows_affected on the same line with Rows_sent and Rows_examined > (ulong) thd->thread_id, (thd->db ? thd->db : ""), > ((thd->query_plan_flags & QPLAN_QC) ? "Yes" : "No"), > query_time_buff, lock_time_buff, > (ulong) thd->get_sent_row_count(), > - (ulong) thd->get_examined_row_count()) == (size_t) -1) > + (ulong) thd->get_examined_row_count(), > + thd->get_stmt_da()->is_ok() ? > + (ulong) thd->get_stmt_da()->affected_rows() : > + 0) == (size_t) -1) > tmp_errno= errno; > if ((thd->variables.log_slow_verbosity & LOG_SLOW_VERBOSITY_QUERY_PLAN) && > (thd->query_plan_flags & Regards, Sergei

2 1

Re: [Maria-developers] [Commits] Rev 4484: MDEV-4255 MIXED replication: Unique key updates are not marked unsafe, replication fails in lp:~maria-captains/maria/10.0
by Kristian Nielsen 29 Nov '14

29 Nov '14

Sergei Golubchik <serg(a)mariadb.org> writes: > Kristian Nielsen, could you please review this patch? I guess my main question here is, did you think about why you want to) fix this 1) in a post-GA release, 2) at all? One problem with adding new statements as unsafe for statement-based binlogging is that it introduces new warnings in existing applications. This is especially annoying on a slave with binlogging enabled, as every unsafe statement now causes a warning in the slave error log. Thus, it can happen that a security update for a user leaves them with an error log that constantly fills up on a busy slave. It is my understanding that statement-based binlogging (as opposed to mixed or row) is still quite popular, and there have been complaints about these warnings before. Another point is that this patch seems to fix only a corner case of a bigger problem? This particular patch is about non-transactional updates. But updating a unique key can break replication just as well on transactional tables: CREATE TABLE t1 (a INT PRIMARY KEY); INSERT INTO t1 VALUES (2), (3), (4); UPDATE t1 SET a=a-1; Since we do not have deferred constraints, this UPDATE will fail or succeed depending on the order in which rows are updated. So a statement that succeeds andgets binlogged on the master can break on the slave if the rows are updated in a different order. So any reason that you fix one part of UPDATE of unique key but not the other? (I suppose the same problem can also happen with foreign key constraints, though it might take some creativity to come up with an example where an UPDATE will conflict or not with foreign keys depending on row update order). The bug has fix version 5.5, your patch seems to be against 10.0, where do you intend to push it? I would probably have chosen to fix in 10.1, if at all, but I'm not necessarily against 5.5 or 10.0, I'm mostly just asking what the reason is for whichever version was chosen? > I don't particularly like that it explicitly invokes > thd->decide_logging_format() . But there's no way around it, first time > logging format is decided in lock_tables(), basically when tables are just > opened. At that point we don't know what fields will be updated yet. I am not really familiar with how binlogging is decided on the master, have not worked with that code so far. So unfortunately I am not able to say much about whether this is a problem or not. At least the call is still before the actual update of rows starts, so I suppose it should be ok... > At lp:~maria-captains/maria/10.0 > > ------------------------------------------------------------ > revno: 4484 > revision-id: sergii(a)pisem.net-20141116234900-747qdhm9ti9xh070 > parent: sergii(a)pisem.net-20141113161325-mdz5ox7e86iyazss > fixes bug: https://mariadb.atlassian.net/browse/MDEV-4255 > committer: Sergei Golubchik <sergii(a)pisem.net> > branch nick: 10.0 > timestamp: Mon 2014-11-17 00:49:00 +0100 > message: > MDEV-4255 MIXED replication: Unique key updates are not marked unsafe, replication fails I think this needs a better description, especially since this can cause new warnings in slave error logs. Here is a suggestion, but feel free to use something else if you prefer: "MDEV-4255 MIXED replication: Unique key updates are not marked unsafe, replication fails Statement-based replication of UPDATE statements that modify (parts of) unique keys is not safe. If the order of row updates differ between master and slave, the slave can end up modifying a different set of rows before a unique key violation error than what was updated on the master. If the table is non-transactional, then the statement will be partially executed on the master and on the slave, but with different result, causing replication to diverge. This patch marks as unsafe for statement-based replication any UPDATE that modifies a unique key on a non-transactinal table. In mixed-mode replication, this will cause the query to be logged in row mode. In statement-mode replication, this will produce a warning about binlogging of unsafe statement." > === modified file 'sql/share/errmsg-utf8.txt' > --- a/sql/share/errmsg-utf8.txt 2014-11-13 16:13:25 +0000 > +++ b/sql/share/errmsg-utf8.txt 2014-11-16 23:49:00 +0000 > @@ -7111,3 +7111,5 @@ ER_SLAVE_SKIP_NOT_IN_GTID > eng "When using GTID, @@sql_slave_skip_counter can not be used. Instead, setting @@gtid_slave_pos explicitly can be used to skip to after a given GTID position." > ER_TABLE_DEFINITION_TOO_BIG > eng "The definition for table %`s is too big" > +ER_BINLOG_UNSAFE_UPDATE_UNIQUE_NONTRANS > + eng "The statement is unsafe because it updates a UNIQUE key colums in a non-transactional table. This is unsafe because in the case of a conflict a non-predictable part of the table will stay updated." (Adding a new error code in a GA release requires some care, especially if fix-version:5.5 is intended, but I'm sure you're better aware of this than most). > === modified file 'sql/sql_lex.h' > --- a/sql/sql_lex.h 2014-08-07 16:06:56 +0000 > +++ b/sql/sql_lex.h 2014-11-16 23:49:00 +0000 > @@ -1319,6 +1319,12 @@ class Query_tables_list > */ > BINLOG_STMT_UNSAFE_AUTOINC_NOT_FIRST, > > + /** > + INSERT into auto-inc field which is not the first part of composed > + primary key. > + */ > + BINLOG_STMT_UPDATE_UNIQUE_NONTRANS, > + Copy-paste error. Here, you copied the comment of the earlier entry, you surely intended to re-write the comment to describe the newly added entry. Hope this helps, - Kristian.

3 7

Re: [Maria-developers] [Commits] 953dd8d: MDEV-6997 MDEV-6996 MDEV-7002 MDEV-7006 MDEV-7009 MDEV-7012 MDEV-7015 MDEV-6977
by Sergei Golubchik 29 Nov '14

29 Nov '14

Hi, sanja! On Nov 05, sanja(a)mariadb.com wrote: > revision-id: 953dd8d87fea224e26d11452eb458c79ba098f4a > parent(s): d6981391eb8f904135dfdedb5b90a7592b35e648 > committer: Oleksandr Byelkin > branch nick: work-maria-10.1-sysvar4 > timestamp: 2014-11-05 11:05:01 +0100 > message: > > MDEV-6997 MDEV-6996 MDEV-7002 MDEV-7006 MDEV-7009 MDEV-7012 MDEV-7015 MDEV-6977 I'd prefer a slightly more verbose commit comment. And don't forget - first line is the subject. For example: Use NO_SET_STMT flag for more variables Fixes MDEV-6997 MDEV-6996 MDEV-7002 MDEV-7006 MDEV-7009 MDEV-7012 MDEV-7015 MDEV-6977 I won't comment on individual variables here, as you're simply disabling them all. See my comments in Jira. Regards, Sergei

1 0

[Maria-developers] MDEV-7004 review
by Sergei Golubchik 28 Nov '14

28 Nov '14

Hi, Sergey I didn't review MDEV-7200 (InnoDB) and patches that are already in 10.1. Almost everything else was fine, see few comments below. > ------------------------------------------------------------ > revno: 4513 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0-power > timestamp: Fri 2014-11-07 22:03:18 +0400 > message: > Multi-rwlock lock for table definition cache hash. > added: > modified: > diff: > === modified file 'sql/table_cache.cc' > --- sql/table_cache.cc 2014-10-21 12:20:39 +0000 > +++ sql/table_cache.cc 2014-11-07 18:03:18 +0000 > @@ -53,6 +53,77 @@ > #include "table.h" > #include "sql_base.h" > > + > +/** > + Table definition hash lock. > + > + Lock consists of multiple rw-locks (8 by default). Lock is write locked > + when all instances are write locked. Lock is read locked when any instance > + is read locked. > + > + This class was implemented to workaround scalability bottleneck of table > + definition hash rw-lock. > + > + To be replaced with lock-free hash when we understand how to solve the > + following problems: > + - lock-free hash iterator (can be workarounded by introducing global > + all_tables list); I don't particularly like that solution. Could you try to switch to lock-free hash now? I'll try to do the iterator asap > + - lf_hash_search() may fail because of OOM. It is rather complex to handle > + it properly. > +*/ > + > +class TDC_lock > +{ > + uint m_instances; > + bool m_write_locked; > + char pad[128]; > + struct st_locks > + { > + mysql_rwlock_t lock; > + char pad[128]; In these cases I usually do: pad[128 - sizeof(mysql_rwlock_t)]; > + } *locks; > +public: > + int init(PSI_rwlock_key key, uint instances) > + { > + m_instances= instances; > + m_write_locked= false; > + locks= (st_locks *) my_malloc(sizeof(struct st_locks) * m_instances, MYF(MY_WME)); > + if (!locks) > + return 1; > + for (uint i= 0; i < m_instances; i++) > + mysql_rwlock_init(key, &locks[i].lock); > + return 0; > + } > + void destroy(void) > + { > + for (uint i= 0; i < m_instances; i++) > + mysql_rwlock_destroy(&locks[i].lock); > + my_free(locks); > + } > + void wrlock(void) > + { > + for (uint i= 0; i < m_instances; i++) > + mysql_rwlock_wrlock(&locks[i].lock); > + m_write_locked= true; > + } > + void rdlock(THD *thd= 0) > + { > + mysql_rwlock_rdlock(&locks[thd ? thd->thread_id % m_instances : 0].lock); > + } > + void unlock(THD *thd= 0) > + { > + if (m_write_locked) > + { > + m_write_locked= false; > + for (uint i= 0; i < m_instances; i++) > + mysql_rwlock_unlock(&locks[i].lock); > + } > + else > + mysql_rwlock_unlock(&locks[thd ? thd->thread_id % m_instances : 0].lock); > + } > +}; > + > + > /** Configuration. */ > ulong tdc_size; /**< Table definition cache threshold for LRU eviction. */ > ulong tc_size; /**< Table cache threshold for LRU eviction. */ > ------------------------------------------------------------ > revno: 4512 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0-power > timestamp: Fri 2014-10-31 15:44:36 +0400 > message: > Preallocate locks on THD mem_root to avoid expensive malloc. > modified: > diff: > === modified file 'sql/lock.cc' > --- sql/lock.cc 2014-09-30 17:31:14 +0000 > +++ sql/lock.cc 2014-10-31 11:44:36 +0000 > @@ -700,7 +699,7 @@ MYSQL_LOCK *get_lock_data(THD *thd, TABL > TABLE **to, **table_buf; > DBUG_ENTER("get_lock_data"); > > - DBUG_ASSERT((flags == GET_LOCK_UNLOCK) || (flags == GET_LOCK_STORE_LOCKS)); > + DBUG_ASSERT((flags & GET_LOCK_UNLOCK) || (flags & GET_LOCK_STORE_LOCKS)); Not the same, old assert also verified that these flags aren't set both. New assert doesn't. But I don't think it matters much. > DBUG_PRINT("info", ("count %d", count)); > > for (i=lock_count=table_count=0 ; i < count ; i++) > ------------------------------------------------------------ > revno: 4508 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0 > timestamp: Tue 2014-10-21 16:20:39 +0400 > message: > modified: > diff: > === modified file 'sql/handler.cc' > --- sql/handler.cc 2014-11-13 10:01:31 +0000 > +++ sql/handler.cc 2014-10-21 12:20:39 +0000 > @@ -410,13 +410,16 @@ static int hton_ext_based_table_discover > static void update_discovery_counters(handlerton *hton, int val) > { > if (hton->discover_table_existence == full_discover_for_existence) > - my_atomic_add32(&need_full_discover_for_existence, val); > + my_atomic_add32_explicit(&need_full_discover_for_existence, val, > + MY_MEMORY_ORDER_RELAXED); > > if (hton->discover_table_names) > - my_atomic_add32(&engines_with_discover_table_names, val); > + my_atomic_add32_explicit(&engines_with_discover_table_names, val, > + MY_MEMORY_ORDER_RELAXED); > > if (hton->discover_table) > - my_atomic_add32(&engines_with_discover, val); > + my_atomic_add32_explicit(&engines_with_discover, val, > + MY_MEMORY_ORDER_RELAXED); I wouldn't do that. Technically there's a dependency between these counters and loaded engines, so the order, kind of, matters. I don't think there is any practical risk of bugs here, but, on the other hand, these counters are only updated when an engine is loaded or unloaded, so there's no need to bother with the relaxed memory ordering here. > } > > int ha_finalize_handlerton(st_plugin_int *plugin) > > === modified file 'sql/mysqld.cc' > --- sql/mysqld.cc 2014-11-18 21:25:47 +0000 > +++ sql/mysqld.cc 2014-10-21 12:20:39 +0000 > @@ -3884,7 +3884,7 @@ static void my_malloc_size_cb_func(long > } > // workaround for gcc 4.2.4-1ubuntu4 -fPIE (from DEB_BUILD_HARDENING=1) > int64 volatile * volatile ptr=&global_status_var.memory_used; > - my_atomic_add64(ptr, size); > + my_atomic_add64_explicit(ptr, size, MY_MEMORY_ORDER_RELAXED); good! > } > } > > > === modified file 'sql/mysqld.h' > --- sql/mysqld.h 2014-11-13 08:56:28 +0000 > +++ sql/mysqld.h 2014-10-21 12:20:39 +0000 > @@ -629,7 +629,7 @@ inline __attribute__((warn_unused_result > { > query_id_t id; > my_atomic_rwlock_wrlock(&global_query_id_lock); > - id= my_atomic_add64(&global_query_id, 1); > + id= my_atomic_add64_explicit(&global_query_id, 1, MY_MEMORY_ORDER_RELAXED); I believe this is fine > my_atomic_rwlock_wrunlock(&global_query_id_lock); > return (id); > } > === modified file 'sql/table_cache.cc' > --- sql/table_cache.cc 2014-06-10 18:20:33 +0000 > +++ sql/table_cache.cc 2014-10-21 12:20:39 +0000 > @@ -139,7 +139,7 @@ uint tc_records(void) > { > uint count; > my_atomic_rwlock_rdlock(&LOCK_tdc_atomics); > - count= my_atomic_load32(&tc_count); > + count= my_atomic_load32_explicit(&tc_count, MY_MEMORY_ORDER_RELAXED); I'm not sure about this file. You should know better whether it's safe to use relaxed memory ordering here. > my_atomic_rwlock_rdunlock(&LOCK_tdc_atomics); > return count; > } Regards, Sergei

2 5

[Maria-developers] add a rotate event at begin of relay log make greate help
by nanyi607rao 28 Nov '14

28 Nov '14

Hi kristian, currently, a relaylog is immediately purged after it be completely replayed (relay_log_purge=1). and if IO thread crashes, we should delete all relay logs and fetch master binlog again from exec_master_log_pos/file (relay_log_purge=ON). I think that make a greap waste of disk IO especially to cloud disk. The critical problem is that there is no method to know the original binlog filename on master of a event in relaylog. Actually we can know a relaylog event's original binlog offset on master, but we can't know the original binlog filename. so why not forcely add a rotate event or any other new type event which contained the original master binlog filename at begin of a relay log. as I know two different master binlogs' events wouldn't contained in one relay log, so the original binlog filename of a relaylog's events is only one. If we want to know a relaylog event's original binlog filename/position, we can get original filename from the begin of relaylog, and get original binlog offset from event's "end log pos". oppositely, if we know a event's binlog filename/end positon on master, wo also can quickly find it in slave's relay log. That can make many benefits, firstly no need to delete all relay logs and fetch master binlogs again when IO thread crashed, because we can get exact last read_master_log_filename/position from last relay log. secondly, there is no need to use GTID in 1 vs n replication failover Scenario (Gtid must set log_slave_updates=ON in mysql 5.6, which increase disk IO load), if master crashes, other slaves can get lost events for the newest slave's relay log, as long as relay log don't be purged, then promotes the newest slave to master. Thanks. 2014-11-26 nanyi607rao

3 3

[Maria-developers] MDEV-6431 review
by Sergey Vojtovich 27 Nov '14

27 Nov '14

Hi Sergei, overall it looks good. I want to double check the clean-up stuff, since I didn't understand everything at the first glance. Some comments inline. > diff --git a/include/mysql/plugin_password_validation.h b/include/mysql/plugin_password_validation.h > new file mode 100644 > index 0000000..0bbb8f4 > --- /dev/null > +++ b/include/mysql/plugin_password_validation.h > @@ -0,0 +1,45 @@ > +/** > + Password validation plugin descriptor > +*/ > +struct st_mysql_password_validation > +{ > + int interface_version; /**< version plugin uses */ > + /** > + Function provided by the plugin which should perform password validation > + and return 0 if the password has passed the validation. > + */ > + int (*validate_password)(MYSQL_LEX_STRING *username, > + MYSQL_LEX_STRING *password); > +}; > +#endif > + This differs from MySQL API, so we can't reuse their plugins. Also they have this cool get_password_strength method, but that's something we can expose via UDF. OTOH we can compare password against username, and that's quite common password misuse. And what I always hated in this validation stuff is that you'll never come up with a password that makes it happy. Of course unless you know requirements. In our case requirements are exposed via system variables. This is more or less acceptable. But I believe it would be nice if this stuff would be able to generate strong passwords. Again, that's something that can be exposed as UDF. > diff --git a/sql/sql_plugin.cc b/sql/sql_plugin.cc > index 529a795..cdf8c7b 100644 > --- a/sql/sql_plugin.cc > +++ b/sql/sql_plugin.cc > @@ -1051,7 +1055,8 @@ static bool plugin_add(MEM_ROOT *tmp_root, > continue; // invalid plugin type > > if (plugin->type == MYSQL_UDF_PLUGIN || > - (plugin->type == 8 && tmp.plugin_dl->mariaversion == 0)) > + (plugin->type == MariaDB_PASSWORD_VALIDATION_INTERFACE_VERSION && > + tmp.plugin_dl->mariaversion == 0)) > continue; // unsupported plugin type > > if (name->str && my_strnncoll(system_charset_info, You compare type against version here. Why? Regards, Sergey

2 1

Re: [Maria-developers] [Commits] Rev 4369: MDEV-7148 - Recurring: InnoDB: Failing assertion: !lock->recursive in lp:maria/5.5
by Kristian Nielsen 27 Nov '14

27 Nov '14

Sergey Vojtovich <svoj(a)mariadb.org> writes: > revno: 4369 > revision-id: svoj(a)mariadb.org-20141120094823-fozdsrm1kzv46kxi > parent: jplindst(a)mariadb.org-20141119182734-cwbaed0ka90ocj5e > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 5.5 > timestamp: Thu 2014-11-20 13:48:23 +0400 > message: > MDEV-7148 - Recurring: InnoDB: Failing assertion: !lock->recursive > > On PPC64 high-loaded server may crash due to assertion failure in InnoDB > rwlocks code. > > This happened because load order between "recursive" and "writer_thread" > wasn't properly enforced. Looks ok. I failed to understand why the patch would fix the assertion mentioned in the bug report. But the patch itself looks correct. - Kristian.

3 3

Re: [Maria-developers] [Commits] Rev 3902: MDEV-7123: MariaDB 10.0.14 Galera node shutdown with signal 11 in lp:~maria-captains/maria/maria-10.0-galera
by Kristian Nielsen 26 Nov '14

26 Nov '14

Nirbhay Choubey <nirbhay(a)mariadb.com> writes: > revno: 3902 > revision-id: nirbhay(a)mariadb.com-20141123163533-9ov1yirj26cur301 > parent: nirbhay(a)mariadb.com-20141119183337-n5spskf54beqbhif > committer: Nirbhay Choubey <nirbhay(a)mariadb.com> > branch nick: maria-10.0-galera > timestamp: Sun 2014-11-23 11:35:33 -0500 > message: > MDEV-7123: MariaDB 10.0.14 Galera node shutdown with signal 11 > > wsrep-patch uses same connection name for constructing Master_info > objects. As a result all existing wsrep Master_info objects refer > to same rpl_filter object. This could lead to race when multiple > threads try to destruct/delete Master_info object, as they would all > try to delete the same relay_log object. > Fixed by adding a mutex to protect the critical section. The patch is ok. But please consider the following points, and act on them or not as you see fit: 1. You are introducing a mutex lock/unlock in a general data structure used in a couple of places (looks like keycaches and rpl_filters?). Did you consider the performance impact of this? When will the additional locking/unlocking take place? If it only happens in less often used operations (like re-configuring replication filters or key caches?), then extra locking is not an issue. If it happens often, like once per query or something like that, it would probably be best to find a solution with less performance impact. (mutex lock and unlock are quite expensive even if not contended). 2. Consider using explicit mysql_mutex_lock() and mysql_mutex_unlock() calls rather than introducing extra lock()/unlock() methods. The extra methods are needless abtractions that make it harder to understand exactly what is happening wrt. locking. 3. Did you consider if a test case could be made for the originally reported bug? We used to have a rule that any bug fix must include a test case that reproduces the bug fixed (ie. the test case must fail before the patch). Not a hard rule (a few bugs can be extremely hard to write mtr testcases for), but a good rule nevertheless. But maybe this rule has been dropped, now that the focus is on quantity of Jira tickes closed rather than quality. - Kristian.

1 0

[Maria-developers] Windows: git config autocrlf
by Alexander Barkov 26 Nov '14

26 Nov '14

Hi, What would be the correct value for the autocrlf setting in git on Windows? Git suggests autocrlf=true by default at installation time. But then mysql-test do not work with this value. What is a better choice between the other two: "false" or "input"? Thanks.

1 0

[Maria-developers] doubt about read uncommited
by Roberto Spadim 25 Nov '14

25 Nov '14

hi guys, i use read uncommited sometimes with innodb, that's nice and work but now, i'm using a myisam table, and a aria table.... does read uncommited work with this kind of engine? i tested and table stay 'waiting table lock' while a looooong update occurs i don't know what's the internal diference but is possible to "easily" implement read uncommited to myisam/aria? -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

4 10

[Maria-developers] Please review: MDEV-7199 Introduce the concept of data type handlers
by Alexander Barkov 25 Nov '14

25 Nov '14

Hi Sergei, while working on addressing your review suggestions for "MDEV-4912 Add a plugin to field types (column types)", I decided to create a separate tasks for a piece of additional changes: https://mariadb.atlassian.net/browse/MDEV-7199 MDEV-7199 Introduce the concept of data type handlers Please review. Thanks.

1 0

Re: [Maria-developers] [Commits] Rev 4507: MDEV-7157 plugins.server_audit fails sporadically in buildbot. in file:///home/hf/wmar/mdev-7157/
by Kristian Nielsen 24 Nov '14

24 Nov '14

<holyfoot(a)askmonty.org> writes: > ------------------------------------------------------------ > revno: 4507 > revision-id: holyfoot(a)askmonty.org-20141123225345-p2xd7yrklzee10u1 > parent: sergii(a)pisem.net-20141121192039-d0lv6cj96kg5pw02 > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: mdev-7157 > timestamp: Mon 2014-11-24 02:53:45 +0400 > message: > MDEV-7157 plugins.server_audit fails sporadically in buildbot. > Records can get to the different place in the log when multiple thread > are logged. So the delay added to let the record be saved on the same > place. > === modified file 'mysql-test/suite/plugins/t/server_audit.test' > --- a/mysql-test/suite/plugins/t/server_audit.test 2014-03-26 19:58:27 +0000 > +++ b/mysql-test/suite/plugins/t/server_audit.test 2014-11-23 22:53:45 +0000 > @@ -43,6 +43,7 @@ show variables like 'server_audit%'; > set global server_audit_mode=1; > set global server_audit_events=''; > create database sa_db; > +--sleep 2 The correct way to do this is to use DEBUG_SYNC... - Kristian.

1 0

Re: [Maria-developers] [Commits] Rev 3879: MDEV-6593 : domain_id based replication filters in lp:~maria-captains/maria/maria-10.0-galera
by Kristian Nielsen 24 Nov '14

24 Nov '14

Nirbhay Choubey <nirbhay(a)skysql.com> writes: > ------------------------------------------------------------ > revno: 3879 > revision-id: nirbhay(a)skysql.com-20140912181622-s33e7r0vtgpju95q > parent: nirbhay(a)skysql.com-20140814224304-rtkeal7p9lk06li1 > committer: Nirbhay Choubey <nirbhay(a)skysql.com> > branch nick: maria-10.0-galera_6593 > timestamp: Fri 2014-09-12 14:16:22 -0400 > message: > MDEV-6593 : domain_id based replication filters > > Added support for IGNORE_DOMAIN_IDS and DO_DOMAIN_IDS. Here is my review. Some general comments, followed by some detailed comments in-line in the patch: 1. Did you prepare any documentation? You could either add it to the knowledgebase directly, or write up a text file and ask Daniel to add it in relevant places. 2. Did you run the full test suite? I would have expected at least some tests in the multi_source suite to need modification. The chance touches enough things that you should test it in a feature tree in Buildbot to ensure that it is green before pushing it to a main tree. 3. I think some extra test cases are needed: - You should test (and document) what happens when the same domain_id is in both the DO and the IGNORE list. Is it actually ever sensible to have both DO_DOMAIN_IDS and IGNORE_DOMAIN_IDS ? If not, I think it should be an error to try. - You should also add a test using non-GTID mode. - You should also add a test where parallel replication is used (--slave-parallel-threads > 0). - Add a test for the following scenario when using GTID mode: Some domain id is ignored, some events are received from the master and ignored. Then the slave is stopped, and the ignore rule is removed. What happens when the slave is restarted and reconnects? I think the previously ignored domain id events will now be re-fetched and replicated. I guess that's ok, but it should be documented. Also test what happens when binlog files on the master have been rotated and purged, so that the events are no longer available on the master (an error should occur). - Add a similar test for when not using GTID mode, the last event replicated by the slave is ignored, the slave is stopped, the ignore rule removed and the slave restarted; will the last event now be re-fetched or not (I think not). - Add a test that checks that seconds_behind_master is properly updated to zero even when the last event is ignored due to DO_DOMAIN_IDS/IGNORE_DOMAIN_IDS. It can perhaps be a bit tricky to test, but I think you can use this existing test as a basis: mysql-test/suite/rpl/t/rpl_parallel2.test - It would also be good to check that the code works when used on an old master (which does not send GTID events). You can do this by creating a binlog file with a 5.5 server and storing it in mysql-test/std_data/, and then installing it in a master; there should be several other test cases that already do this. 4. Also see detailed comments for some possible problems with the implementation. The most serious is probably to ensure that events are not skipped after the end of the group, we need a couple of tests for this, see below. Also, there is a small memory leak, see below. > +DO_DOMAIN_IDS (before) : 1 > +IGNORE_DOMAIN_IDS (before) : 2 > +CHANGE MASTER TO DO_DOMAIN_IDS=(4,4,5,1,7,7,7,1,1,2,6,8,1,4,5,5,9,3), MASTER_USE_GTID=current_pos; So if a domain_id is specified multiple times, the redundant ones will be ignored. It would seem preferable to give an error... but as far as I can see the behaviour is the same for IGNORE_SERVER_IDS, so better to be consistent as you did in the code. Just be sure to add this to the documentation. > === added file 'mysql-test/suite/rpl/t/rpl_domain_id_filter.test' > --- a/mysql-test/suite/rpl/t/rpl_domain_id_filter.test 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/suite/rpl/t/rpl_domain_id_filter.test 2014-09-12 18:16:22 +0000 > +SET @gtid_domain_id_saved = @@session.gtid_domain_id; This is not needed, nor do you use it anywhere, you can remove it. > +##### Case 3: When both DO_DOMAIN_IDS and IGNORE_DOMAIN_IDS are non-empty > + > +source include/stop_slave.inc; > +let $do_domain_ids_before= query_get_value(SHOW SLAVE STATUS, Replicate_Do_Domain_Ids, 1); > +let $ignore_domain_ids_before= query_get_value(SHOW SLAVE STATUS, Replicate_Ignore_Domain_Ids, 1); > +--echo DO_DOMAIN_IDS (before) : $do_domain_ids_before > +--echo IGNORE_DOMAIN_IDS (before) : $ignore_domain_ids_before > + > +# Replicate events belonging to "domain_id 1". (DO_DOMAIN_IDS should win!) > +CHANGE MASTER TO DO_DOMAIN_IDS=(1), IGNORE_DOMAIN_IDS=(2), MASTER_USE_GTID=current_pos; Did you intend here to test overlapping ids in DO_DOMAIN_IDS and IGNORE_DOMAIN_IDS (due to the comment "DO_DOMAIN_IDS should win") ? But if I understand correctly, IGNORE_DOMAIN_IDS will be ignored if DO_DOMAIN_IDS is non-empty. So I think it should be an error to try to set both - do you agree? > +--source include/rpl_end.inc Please add additional testcases, as suggested at the start of review. > === modified file 'sql/rpl_mi.cc' > --- a/sql/rpl_mi.cc 2014-02-19 10:05:15 +0000 > +++ b/sql/rpl_mi.cc 2014-09-12 18:16:22 +0000 > @@ -513,6 +513,25 @@ > else > mi->using_gtid= Master_info::USE_GTID_NO; > } > + > + // DO_DOMAIN_IDS > + if (0 == strncmp(buf, STRING_WITH_LEN("do_domain_ids=")) && > + mi->domain_id_filter.init_ids(buf + sizeof("do_domain_ids"), > + Domain_id_filter::DO_DOMAIN_IDS)) > + { > + sql_print_error("Failed to initialize master info do_domain_ids"); > + goto errwithmsg; > + } > + > + // IGNORE_DOMAIN_IDS > + if (0 == strncmp(buf, STRING_WITH_LEN("ignore_domain_ids=")) && > + mi->domain_id_filter.init_ids(buf + sizeof("ignore_domain_ids"), > + Domain_id_filter::IGNORE_DOMAIN_IDS)) > + { > + sql_print_error("Failed to initialize master info " > + "ignore_domain_ids"); > + goto errwithmsg; > + } So here you use the same style as adjacent code (that I wrote). However, I have since learned that the style (0 == strncmp(...)) is not used in the server. So please change this, as well as my existing code, to use either strncmp(...) == 0 or !strncmp(), as you prefer. Another thing from my old code is the sizeof("do_domain_ids"), which highly confused me for a bit before I realised that while the '=' is not included in the sizeof(), the terminating '\0' _is_ included. So since even I myself is confused by this code, better clarify it ;-) Perhaps it would be clearer if changed to (sizeof("do_domain_ids=")-1) (both in my old code and in your new code). (Normally it's good to follow the style of existing code as you did, I just noticed this in the review and realised that my own "existing code" would benefit from these small improvements). > @@ -634,6 +653,16 @@ > } > } > > + char* do_domain_ids_buf= > + mi->domain_id_filter.as_string(Domain_id_filter::DO_DOMAIN_IDS); > + if (do_domain_ids_buf == NULL) > + DBUG_RETURN(1); > + > + char* ignore_domain_ids_buf= > + mi->domain_id_filter.as_string(Domain_id_filter::IGNORE_DOMAIN_IDS); > + if (ignore_domain_ids_buf == NULL) > + DBUG_RETURN(1); You have a memory leak here, do_domain_ids_buf will not be freed if ignore_domain_ids_buf fails due to out-of-memory. > @@ -1348,4 +1382,218 @@ > DBUG_RETURN(result); > } > > +/* ctor */ > +Domain_id_filter::Domain_id_filter() : m_filter(false) > +{ > + for (int i= DO_DOMAIN_IDS; i <= IGNORE_DOMAIN_IDS; i ++) > + { > + my_init_dynamic_array(&m_domain_ids[i], sizeof(uint32), 16, 16, MYF(0)); > + } > +} > + > +/* dtor */ > +Domain_id_filter::~Domain_id_filter() I don't think comments like /* ctor */ and /* dtor */ are useful, better just omit them. > +/** > + Update the domain id list of the given type with elements from the new list. > + > + @param new_ids [IN] new list of ids > + @param type [IN] domain id list type to update > + @param changed [IN] has the domain id list changed in the > + last CHANGE MASTER command? > + > + @retval void > +*/ > +void Domain_id_filter::update_ids(DYNAMIC_ARRAY *new_ids, enum_list_type type, > + bool changed) > +{ > + DYNAMIC_ARRAY *ids= &m_domain_ids[type]; > + > + if (changed == true) > + reset_dynamic(ids); > + > + /* bsearch requires an ordered list. */ > + sort_dynamic(new_ids, change_master_domain_id_cmp); > + > + for (uint i= 0; i < new_ids->elements; i++) > + { > + uint32 domain_id; > + get_dynamic(new_ids, (void *) &domain_id, i); > + > + if (bsearch((const uint32 *) &domain_id, ids->buffer, ids->elements, > + sizeof(uint32), change_master_domain_id_cmp) == NULL) > + { > + insert_dynamic(ids, (uint32*) &domain_id); > + } > + } > + return; > +} Ok, so I understand that this code is modelled after the existing code for IGNORED_SERVER_IDS. But I found it rather confusing, because it looks like there is a bug: if changed==false, then the code does not ensure that the resulting array will still be sorted. I guess the bug is not possible to trigger, because the list of stuff to add will only be non-empty if changed==true? But that is rather confusing. Please simplify the logic; maybe you only need to call this function when changed==true, so the parameter can be omitted and the array always reset? Also, consider if you can rewrite the existing code for IGNORE_SERVER_IDS so that the same common code can be used as for IGNORE_DOMAIN_IDS and DO_DOMAIN_IDS, that would simplify the code and reduce risk of bugs or inconsistencies. > +/** > + Initialize the given domain id list with the numbers from the specified > + buffer. > + > + @param buf [IN] buffer > + @param type [IN] domain id list type > + > + @retval false Success > + true Error > +*/ > +bool Domain_id_filter::init_ids(char *buf, enum_list_type type) > +{ > + char *token, *last; > + uint num_items; > + DYNAMIC_ARRAY *ids= &m_domain_ids[type]; > + > + token= strtok_r(buf, " ", &last); > + if (token == NULL) > + { > + return true; /* error */ > + } > + > + num_items= atoi(token); > + for (uint i= 0; i < num_items; i++) > + { > + token= strtok_r(NULL, " ", &last); > + if (token == NULL) > + { > + return true; /* error */ > + } > + else > + { > + uint32 val= atoi(token); > + insert_dynamic(ids, (uchar *) &val); > + } > + } > + return false; > +} Again, did you consider using the same code for this and for IGNORE_SERVER_IDS? (I think you did, because you added a comment to init_dynarray_intvar_from_file ;-). Maybe consider it again; it is fine to rewrite/remove the existing code for IGNORE_SERVER_IDS and replace it with your new code. Again, it would reduce risk for bugs or inconsistencies. > + /* getter */ > + bool get_filter() { return m_filter; } > + > + /* setter */ > + void set_filter(uint32 domain_id); These are not really getter and setters, I think, at least I find those names confusing. Maybe something like do_filter(uint32 domain_id) and is_group_filtered() ? And expand the comments, so that they explain that these functions respectively check a GTID for needing to be filtered, and return whether the current group is being filtered. Also, personally I dislike setter/getter, and would prefer just to access the m_filter as a public member. But I suppose that's style and up to you. > + /* > + Initialize the given domain id list with the numbers from the specified > + buffer. > + > + @param buf [IN] buffer > + @param type [IN] domain id list type > + > + @retval false Success > + true Error > + */ > + bool init_ids(char *buf, enum_list_type type); Clarify in the comment that the list is initialized from a string of space-separated numbers, the first of which is the number of following entries. > @@ -182,6 +279,7 @@ > bool flush_relay_log_cache, > bool need_lock_relay_log); > int change_master_server_id_cmp(ulong *id1, ulong *id2); > +int change_master_domain_id_cmp(uint32 *id1, uint32 *id2); I think change_master_domain_id_cmp() is only used in rpl_mi.cc, so should be static there, and not refered here in rpl_mi.h > === modified file 'sql/slave.cc' > --- a/sql/slave.cc 2014-08-12 03:55:41 +0000 > +++ b/sql/slave.cc 2014-09-12 18:16:22 +0000 > @@ -1259,6 +1259,7 @@ > going to ignore (to not log them in the relay log). > Items being read are supposed to be decimal output of values of a > type shorter or equal of @c long and separated by the single space. > + It also used to restore DO_DOMAIN_IDS & IGNORE_DOMAIN_IDS lists. This comment suggests you wanted to share code, then later changed your mind. If you end up keeping the code separate, you still need to delete this comment. (And there is a typo: "It also" -> "It is also", but if you remove it anyway...). > @@ -5601,6 +5610,10 @@ > mi->last_queued_gtid= event_gtid; > mi->last_queued_gtid_standalone= > (gtid_flag & Gtid_log_event::FL_STANDALONE) != 0; > + > + /* Should filter all the subsequent events in the current GTID group? */ > + mi->domain_id_filter.set_filter(event_gtid.domain_id); So, here we will set m_filter to true, if we want to skip the following group. But I don't see any code that will set it to false at the end of the group? That seems a problem, as we could then end up wrongly skipping all kinds of other events, such as Gtid_list, Incident, ... Also note that it is possible for a GTID group to not be terminated by COMMIT/XID. This can happen at least if the master crashes in the middle of writing a group; the slave will receive only the first part of it, followed by a master restart Format_description event. That case needs to be handled, and you should add a test case for it. Also it is possible for eg. a master downgrade to introduce following event groups that do not start with GTID - seems these would also be wrongly skipped? So we also need a test case for this... Hope this helps, - Kristian.

2 8

[Maria-developers] The variable "table_definition_cache" is autosized by default on MySQL 5.6.8+ but not on MariaDB 10
by Jean Weisbuch 20 Nov '14

20 Nov '14

The value of the variable "table_definition_cache" is by default autosized (based on the table_open_cache value) on MySQL >=5.6.8 while it is still static (400) on the latest MariaDB 10.0.14 and 10.1.?, is that by choice or this feature hasn't been backported and should be? From the MySQL doc http://dev.mysql.com/doc/refman/5.6/en/server-system-variables.html#sysvar_… : *Permitted Values (>= 5.6.8)* *Type* |numeric| *Default* |-1 (autosized)| *Min Value* |400| *Max Value* |524288| The default value is based on the following formula, capped to a limit of 2000: 400 + (table_open_cache / 2) Before MySQL 5.6.8, the default is 400.

1 0

Re: [Maria-developers] [Commits] Rev 4367: MDEV-7026: Race in InnoDB/XtraDB mutex implementation can stall or hang the server. in http://bazaar.launchpad.net/~maria-captains/maria/5.5
by Sergey Vojtovich 20 Nov '14

20 Nov '14

Hi Kristian, thanks for taking over this bug! I believe this is the patch that was pushed into 5.5. Unfortunately I think there is still one potential for a hang left (details below). :( Since InnoDB and XtraDB changes are identical I'll comment only XtraDB changes. So you removed os_isync and implemented os_mb instead. That's good! You also removed a bunch of os_wmb/os_rmb calls. That's good too. And you kept RELEASE barrier for mutex_exit() on PPC64. Very good! I will also comment things that I don't really like, but they're just for your information (no action required). On Wed, Nov 19, 2014 at 01:56:46PM +0100, knielsen(a)knielsen-hq.org wrote: > At http://bazaar.launchpad.net/~maria-captains/maria/5.5 > > ------------------------------------------------------------ > revno: 4367 > revision-id: knielsen(a)knielsen-hq.org-20141119125646-njq02h7irmu9s71b > parent: sergii(a)pisem.net-20141118231952-fk3o69075dn3ravv > committer: Kristian Nielsen <knielsen(a)knielsen-hq.org> > branch nick: mariadb-5.5 > timestamp: Wed 2014-11-19 13:56:46 +0100 > message: > MDEV-7026: Race in InnoDB/XtraDB mutex implementation can stall or hang the server. > > The bug was that full memory barrier was missing in the code that ensures that > a waiter on an InnoDB mutex will not go to sleep unless it is guaranteed to be > woken up again by another thread currently holding the mutex. This made > possible a race where a thread could get stuck waiting for a mutex that is in > fact no longer locked. If that thread was also holding other critical locks, > this could stall the entire server. There is an error monitor thread than can > break the stall, it runs about once per second. But if the error monitor > thread itself got stuck or was not running, then the entire server could hang > infinitely. > > This was introduced on i386/amd64 platforms in 5.5.40 and 10.0.13 by an > incorrect patch that tried to fix the similar problem for PowerPC. JFYI: Strictly speaking that "incorrect patch" attempted to fix different problem (read carefully): it added RELEASE barrier between "stores inside mutex" and "lock_word= 0" while this patch adds full barrier between "lock_word= 0" and "if (waiters)". > > This commit reverts the incorrect PowerPC patch, and instead implements a fix > for PowerPC that does not change i386/amd64 behaviour, making PowerPC work > similarly to i386/amd64. ...skip... > === modified file 'storage/xtradb/include/os0sync.h' > --- a/storage/xtradb/include/os0sync.h 2014-09-08 15:10:48 +0000 > +++ b/storage/xtradb/include/os0sync.h 2014-11-19 12:56:46 +0000 > @@ -314,11 +331,28 @@ amount of increment. */ > /**********************************************************//** > Returns the old value of *ptr, atomically sets *ptr to new_val */ > > -# define os_atomic_test_and_set_byte(ptr, new_val) \ > +#ifdef __powerpc__ > +/* > + os_atomic_test_and_set_byte_release() should imply a release barrier before > + setting, and a full barrier after. But __sync_lock_test_and_set() is only > + documented as an aquire barrier. So on PowerPC we need to add the full > + barrier explicitly. */ > +# define os_atomic_test_and_set_byte_release(ptr, new_val) \ > + do { __sync_lock_release(ptr); \ > + __sync_synchronize(); } while (0) > +#else > +/* > + On x86, __sync_lock_test_and_set() happens to be full barrier, due to > + LOCK prefix. > +*/ > +# define os_atomic_test_and_set_byte_release(ptr, new_val) \ > + __sync_lock_test_and_set(ptr, (byte) new_val) > +#endif JFYI: "os_atomic_test_and_set_byte_release" sounds odd to me. Look at gcc manual: https://gcc.gnu.org/onlinedocs/gcc-4.1.2/gcc/Atomic-Builtins.html ... This builtin is not a full barrier, but rather an acquire barrier. This means that references after the builtin cannot move to (or be speculated to) before the builtin, but previous memory stores may not be globally visible yet, and previous memory loads may not yet be satisfied. ... You name RELEASE something that is acqually ACQUIRE. So why didn't you like "os_atomic_lock_release_byte" which is RELASE btw: ... This builtin is not a full barrier, but rather a release barrier. This means that all previous memory stores are globally visible, and all previous memory loads have been satisfied, but following memory reads are not prevented from being speculated to before the barrier. ... > +/* > + os_atomic_test_and_set_byte_acquire() is a full memory barrier on x86. But > + in general, just an aquire barrier should be sufficient. */ > +# define os_atomic_test_and_set_byte_acquire(ptr, new_val) \ > __sync_lock_test_and_set(ptr, (byte) new_val) !!! The problem is here!!! Strictly speaking not here, but in mutex_spin_wait(): ... mutex_set_waiters(mutex, 1); /* Try to reserve still a few times */ for (i = 0; i < 4; i++) { if (mutex_test_and_set(mutex) == 0) { ... We'll get stuck if "waiters= 1" will become visible after "if (mutex_test_and_set(mutex))". I don't see anything preventing this reorder. ...skip... > === modified file 'storage/xtradb/include/sync0sync.ic' > --- a/storage/xtradb/include/sync0sync.ic 2014-08-29 12:02:46 +0000 > +++ b/storage/xtradb/include/sync0sync.ic 2014-11-19 12:56:46 +0000 > @@ -80,11 +80,11 @@ mutex_test_and_set( > mutex_t* mutex) /*!< in: mutex */ > { > #if defined(HAVE_ATOMIC_BUILTINS) > - return(os_atomic_test_and_set_byte(&mutex->lock_word, 1)); > + return(os_atomic_test_and_set_byte_acquire(&mutex->lock_word, 1)); > #else > ibool ret; > > - ret = os_fast_mutex_trylock(&(mutex->os_fast_mutex)); > + ret = os_fast_mutex_trylock_full_barrier(&(mutex->os_fast_mutex)); > > if (ret == 0) { > /* We check that os_fast_mutex_trylock does not leak JFYI: This is probably the most funny thing. Here we're in the "#else" branch of "#if defined(HAVE_ATOMIC_BUILTINS)". E.g. if we compile with gcc and enter this branch it means: sync builtins are not avialble (gcc atomic builtins are very likely not available too). Now you say "os_fast_mutex_trylock_full_barrier" which does os_mb. But since neither sync bultins nor atomic builtins are available, os_mb is basically no-op. This may be different on other platforms, but to my knowledge memory barriers and atomic operations normally come together. Another thing: did you try compiling with this branch enabled? IIRC it didn't compile even berfore we started fixing it. ...skip... Regards, Sergey

2 2

[Maria-developers] constant propagation & LIKE
by Alexander Barkov 20 Nov '14

20 Nov '14

Hello Sergei, The attached patch fixes MDEV-7149 Constant propagation erroneously applied for LIKE All tests pass. But I'm not 100% sure about the patch. Is eq_cmp_result() proper place to fix this? If it's the right place, I can test the collation properties and return: - COND_TRUE for simple collations with one-character-to-one-weight rules, - COND_OK for the complex collations with tricky character to weight mapping. Thanks.

1 0

Re: [Maria-developers] [Commits] Rev 4358: MDEV-7113 difference between check_vcol_func_processor and check_partition_func_processor in lp:~maria-captains/maria/5.5
by Sergey Petrunia 20 Nov '14

20 Nov '14

Hi Sergei, * what about persistent virtual columns? Do we still support environment-dependent virtual functions for those? * the behavior of current solution is not deterministic: thread1> select @@time_zone; +-------------+ | @@time_zone | +-------------+ | SYSTEM | +-------------+ 1 row in set (0.00 sec) thread1> create table t2 ( a varchar(32), b int, v varchar(100) as (from_unixtime(b))); Query OK, 0 rows affected (0.28 sec) thread1> insert into t2 (a,b) values ('local-tz-23-11', unix_timestamp()); Query OK, 1 row affected (0.04 sec) thread1> select * from t2; +----------------+------------+---------------------+ | a | b | v | +----------------+------------+---------------------+ | local-tz-23-11 | 1416427891 | 2014-11-19 23:11:31 | +----------------+------------+---------------------+ 1 row in set (0.00 sec) thread2> set time_zone='Europe/Helsinki'; Query OK, 0 rows affected (0.00 sec) thread2> select * from t2; +----------------+------------+---------------------+ | a | b | v | +----------------+------------+---------------------+ | local-tz-23-11 | 1416427891 | 2014-11-19 23:11:31 | +----------------+------------+---------------------+ 1 row in set (0.00 sec) thread2> flush tables; Query OK, 0 rows affected (0.00 sec) thread2> select * from t2; +----------------+------------+---------------------+ | a | b | v | +----------------+------------+---------------------+ | local-tz-23-11 | 1416427891 | 2014-11-19 22:11:31 | +----------------+------------+---------------------+ 1 row in set (0.00 sec) Is this really ok? Maybe, it makes sense to use some some "default" (e.g. global variables) environment on the non-deterministic virtual column functions? Or, make these functions always fetch the environment from the current thread? On Mon, Nov 17, 2014 at 09:43:46AM +0100, Sergei Golubchik wrote: > At lp:~maria-captains/maria/5.5 > > ------------------------------------------------------------ > revno: 4358 > revision-id: sergii(a)pisem.net-20141115155753-rx5twf6utu1isczh > parent: sergii(a)pisem.net-20141113173729-hirl67og6at3xbni > fixes bugs: https://mariadb.atlassian.net/browse/MDEV-6789 https://mariadb.atlassian.net/browse/MDEV-7113 > committer: Sergei Golubchik <sergii(a)pisem.net> > branch nick: 5.5 > timestamp: Sat 2014-11-15 16:57:53 +0100 > message: > MDEV-7113 difference between check_vcol_func_processor and check_partition_func_processor > MDEV-6789 segfault in Item_func_from_unixtime::get_date on updating table with virtual columns > > * prohibit VALUES in partitioning expression > * prohibit user and system variables in virtual column expressions > * fix Item_func_date_format to cache locale (for %M/%W to return the same as MONTHNAME/DAYNAME) > * fix Item_func_from_unixtime to cache time_zone directly, not THD (and not to crash) > * added tests for other incorrectly allowed (in vcols) functions to see that they don't crash ... BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

2 2

Re: [Maria-developers] [Commits] Rev 4346: MDEV-7026 - Occasional hang during startup on Power8 in lp:maria/5.5
by Kristian Nielsen 19 Nov '14

19 Nov '14

Sergey Vojtovich <svoj(a)mariadb.org> writes: > revno: 4346 > revision-id: svoj(a)mariadb.org-20141118080742-sdf93kgsgxulenvb > parent: knielsen(a)knielsen-hq.org-20141112101013-t63ayylsk08ncgs3 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 5.5 > timestamp: Tue 2014-11-18 12:07:42 +0400 > message: > MDEV-7026 - Occasional hang during startup on Power8 Right, so you requested second review ASAP, I'm trying to understand what's going on here. If my understanding is correct, this patch goes on top of current 5.5.40, not on top of the patch I reviewed previously. So there are two cases, depending on whether INNODB_RW_LOCKS_USE_ATOMICS is set or not. It is my understanding that this is "usually" set, eg. on our common x86 and powerpc linux binaries. The critical thing for the patch at hand is the memory barrier between mutex_set_waiters() and mutex_test_and_set() in the waiter, and mutex_reset_lock_word() and mutex_get_waiters() in the thread releasing the lock. A full memory barrier is needed in the waiter, to be sure that mutex_set_waiters() becomes visible to the releasing thread before mutex_test_and_set() runs. And a full barrier is needed in the releasing thread, to be sure that mutex_reset_lock_word() becomes visible before mutex_get_waiters() runs. So let's see how this is ensured. First the case of x86 with INNODB_RW_LOCKS_USE_ATOMICS set, HAVE_ATOMIC_BUILTINS, pre-5.5.40: - mutex_set_waiters() uses os_compare_and_swap_ulint() which is __sync_bool_compare_and_swap() which is full barrier. - mutex_reset_lock_word() uses os_atomic_test_and_set_byte() which is __sync_lock_test_and_set(). This is documented only as a release barrier, but is in fact a full barrier on x86 with current GCC. So in this case full barrier is there as needed. If HAVE_ATOMIC_BUILTINS is not set, then mutex_reset_lock_word() uses os_fast_mutex_unlock() which is pthread_mutex_unlock(), which is also full barrier on x86. If INNODB_RW_LOCKS_USE_ATOMICS is not set, then mutex_set_waiters() has no barriers. But then mutex_test_and_set() instead has a full barrier on x86. Either in os_atomic_test_and_set_byte() which is __sync_lock_test_and_set(), if HAVE_ATOMIC_BUILTINS is set, or in os_fast_mutex_trylock() otherwise. Both of these are full barriers on x86. So this is why things work ok on x86, apparently. Very bad code that seriously needs cleanup and proper documentation of memory barrier semantics. But it is what we have to work with in 5.5. So to make things work on PowerPC without risk of breaking x86, we could do this: - Make sure os_atomic_test_and_set_byte() is full barrier on PowerPC: #ifdef __powerpc__ /* os_atomic_test_and_set_byte() should imply a full barrier. But __sync_lock_test_and_set() is only documented as an aquire barrier. So on PowerPC we need to add the full barrier explicitly. */ # define os_atomic_test_and_set_byte(ptr, new_val) \ do { __sync_lock_test_and_set(ptr, (byte) new_val); \ __atomic_thread_fence(__ATOMIC_SEQ_CST); } while(0) #else /* On x86, __sync_lock_test_and_set() happens to be full barrier, due to LOCK prefix. */ # define os_atomic_test_and_set_byte(ptr, new_val) \ __sync_lock_test_and_set(ptr, (byte) new_val) #endif - Make sure that mutex_reset_lock_word() is a full barrier when HAVE_ATOMIC_BUILTINS is not set, by adding a memory barrier: os_fast_mutex_unlock(&(mutex->os_fast_mutex)); #ifdef __powerpc__ /* On PowerPC, pthread_mutex does not imply full barrier, so we need to add it explicitly. */ __atomic_thread_fence(__ATOMIC_SEQ_CST); #endif - For the case of neither HAVE_ATOMIC_BUILTINS nor INNODB_RW_LOCKS_USE_ATOMICS set, similarly add a full barrier in mutex_test_and_set() for PowerPC: #ifdef __powerpc__ /* On PowerPC, pthread_mutex does not imply full barrier, so we need to add it explicitly. */ __atomic_thread_fence(__ATOMIC_SEQ_CST); #endif ret = os_fast_mutex_trylock(&(mutex->os_fast_mutex)); Then in 10.1. if someone is brave enough, we can clean this up properly. Clearly define what the memory barrier semantics is, and implement it using the correct primitives that work as needed on all platforms. So now that I understand (hopefully...) how the code used to work, and how it can be fixed for PowerPC without changing existing code, let me look at the new patch ... > === modified file 'storage/innobase/include/os0sync.h' > --- a/storage/innobase/include/os0sync.h 2014-09-08 15:10:48 +0000 > +++ b/storage/innobase/include/os0sync.h 2014-11-18 08:07:42 +0000 > @@ -310,8 +310,16 @@ Returns the old value of *ptr, atomicall > # define os_atomic_test_and_set_byte(ptr, new_val) \ > __sync_lock_test_and_set(ptr, (byte) new_val) > > +#ifdef __powerpc__ > # define os_atomic_lock_release_byte(ptr) \ > __sync_lock_release(ptr) > +#else > +/* In theory __sync_lock_release should be used to release the lock. > +Unfortunately, it does not work properly alone. The workaround is > +that more conservative __sync_lock_test_and_set is used instead. */ > +# define os_atomic_lock_release_byte(ptr) \ > + __sync_lock_test_and_set(ptr, 0) > +#endif This is very confusing. You define a method with full barrier semantics, but call it something suggesting release semantics. Why not revert the original change so that mutex_reset_lock_word() uses os_atomic_test_and_set_byte()? And then fix the implementation of os_atomic_test_and_set_byte() to be full barrier on PowerPC, as I suggested above? The term "atomic" generally implies full barrier in the MySQL source code... > #elif defined(HAVE_IB_SOLARIS_ATOMICS) > > @@ -428,7 +436,7 @@ clobbered */ > # define os_rmb __atomic_thread_fence(__ATOMIC_ACQUIRE) > # define os_wmb __atomic_thread_fence(__ATOMIC_RELEASE) > #ifdef __powerpc__ > -# define os_isync __asm __volatile ("isync":::"memory") > +# define os_isync __atomic_thread_fence(__ATOMIC_SEQ_CST) > #else > #define os_isync do { } while(0) > #endif As Jonas suggested, don't call a full memory barrier "isync". And do not introduce something that is a barrier on one platform but a no-operation on another platform. But ... > === modified file 'storage/innobase/sync/sync0sync.c' > --- a/storage/innobase/sync/sync0sync.c 2014-11-03 13:43:44 +0000 > +++ b/storage/innobase/sync/sync0sync.c 2014-11-18 08:07:42 +0000 > @@ -474,10 +474,9 @@ mutex_set_waiters( > > ptr = &(mutex->waiters); > > - os_wmb; > - > *ptr = n; /* Here we assume that the write of a single > word in memory is atomic */ > + os_isync; > } With this, the semantics of how things work becomes even more complicated. Now on PowerPC the barrier is in one function, but on x86 the barrier is in another function. Why not instead do as I suggested above? Put in full barriers for PowerPC in the same places that they are in x86? ---- Maybe I should try to say if I think the patch fixes the actual bug, in case you don't want to go with my suggestions and instead keep your approach? Since it goes back to using __sync_lock_test_and_set() on x86, and since it has a full barrier in mutex_exit_func() on PowerPC, I think perhaps it does. But it is really hard to understand exactly what is going on, as you can see above I had to do a lot of reasoning to understand even why the old code worked... My advice is still to revert the original incorrect PowerPC patch and do something that is at least consistent with the existing code... (And then feel free to do a proper cleanup in 10.1). Hope this helps, - Kristian.

2 2

Re: [Maria-developers] [Commits] Rev 4346: MDEV-7026 - Occasional hang during startup on Power8 in lp:maria/5.5
by Jonas Oreland 18 Nov '14

18 Nov '14

> > Since isync is not a memory barrier and lwsync doesn't guarantee > StoreLoad > order, the only option we have is full memory barrier. > === modified file 'storage/innobase/include/os0sync.h' > --- a/storage/innobase/include/os0sync.h 2014-09-08 15:10:48 +0000 > +++ b/storage/innobase/include/os0sync.h 2014-11-18 08:07:42 +0000 > > #ifdef __powerpc__ > -# define os_isync __asm __volatile ("isync":::"memory") > +# define os_isync __atomic_thread_fence(__ATOMIC_SEQ_CST) > you have to rename to the #define too (e.g to os_mb). having a os_isync which is infact a mb is more obfuscation than anyone should have to put up with. /Jonas

2 1

Re: [Maria-developers] [Commits] Rev 4346: MDEV-7026 - Occasional hang during startup on Power8 in lp:maria/5.5
by Kristian Nielsen 17 Nov '14

17 Nov '14

Sergey Vojtovich <svoj(a)mariadb.org> writes: > revno: 4346 > revision-id: svoj(a)mariadb.org-20141112114907-thnro6e3kg2ofdsw > committer: Sergey Vojtovich <svoj(a)mariadb.org> > timestamp: Wed 2014-11-12 15:49:07 +0400 > message: > MDEV-7026 - Occasional hang during startup on Power8 > === modified file 'storage/xtradb/sync/sync0sync.c' > --- a/storage/xtradb/sync/sync0sync.c 2014-11-03 13:43:44 +0000 > +++ b/storage/xtradb/sync/sync0sync.c 2014-11-12 11:49:07 +0000 > @@ -467,26 +467,15 @@ mutex_set_waiters( > mutex_t* mutex, /*!< in: mutex */ > ulint n) /*!< in: value to set */ > { > -#ifdef INNODB_RW_LOCKS_USE_ATOMICS > - ut_ad(mutex); > - > - if (n) { > - os_compare_and_swap_ulint(&mutex->waiters, 0, 1); > - } else { > - os_compare_and_swap_ulint(&mutex->waiters, 1, 0); > - } > -#else > volatile ulint* ptr; /* declared volatile to ensure that > the value is stored to memory */ > ut_ad(mutex); > > ptr = &(mutex->waiters); > > - os_wmb; > - > *ptr = n; /* Here we assume that the write of a single > word in memory is atomic */ > -#endif > + os_sync; > } This code is still substantially different from what it was in 5.5.39 and 10.0.12, right? I think we need to instead revert the code to what it was before introducing the problem. The InnoDB mutex code clearly does not work by careful design - mutex_reset_lock_word() needs a full barrier but uses what is documented by GCC as only an acquire barrier. Rather, it works by having been tweaked over the years to just happen to work (eg. GCC in fact uses a full barrier on x86). So changing this to be "correct" is very dangerous in a stable release. It is not possible to ensure correctness by testing, either. But it is not necessary to change it. Instead, leave the code as it was before, and use #ifdef __powerpc__ to ensure that it will work also on PowerPC. For example, something like this (just a sketch, I didn't check exactly how the code should look): #ifdef __powerpc__ /* InnoDB mutex implementation needs full memory barrier for this operation */ # define os_atomic_test_and_set_byte(ptr, new_val) \ do { sync(); __sync_lock_test_and_set(ptr, (byte) new_val); } while (0) #else # define os_atomic_test_and_set_byte(ptr, new_val) \ __sync_lock_test_and_set(ptr, (byte) new_val) #endif Then we leave stable releases 5.5 and 10.0 unchanged, reducing the risk of introducing another serious breakage. And we can still fix PowerPC to work correctly. And then perhaps in 10.1 we can do refactoring of the use of memory barriers in InnoDB, if we really want. (Just in case it's not obvious, yes, I _really_ have the InnoDB mutex code). Remember, the 5.5 and 10.0 minor releases appear as security updates in distros, so it is very important to minimise the risk of introducing serious regressions. > Since isync is not a memory barrier and lwsync doesn't guarantee StoreLoad > order, the only option we have is full memory barrier. It's not a problem to have a full barrier here, I think. Because at this point, the waiter has already gone through a lot of spin loops, and now it is going to take a global mutex on the sync wait array, scan the array, and go to sleep with a context switch. One less full barrier is not going to help a lot. To improve this contended case of the InnoDB mutexes, more radical changes are needed. > InnoDB/XtraDB was getting stuck during server startup. In particular (but not > limited to) while creating doublewrite buffer. The stuck during startup is kind of the minor part of the bug. The major part is stalling the server during operation. (The hang during startup was just how the problem was detected in Buildbot). Better clarify in the commit message that this fixes a bug that could cause the entire server to stall (or even hang completely if error monitor thread itself got stuck). This will help people who experienced the problem and look at the changelog. > It happens because mutex_exit() didn't wake up waiters of affected mutex. This > is kind of acceptable because error monitor thread is supposed to wake up such > waiters. But since error monitor thread hasn't been started that early, nobody > did inform waiting threads that mutex was released. I do not think the error monitor is "supposed to wake up such waiters". In fact, it seems clear that the error monitor wakeup is _not_ used, at least not in commonly used binaries. Because it just never worked - it itself needs to take mutexes, which will prevent it from doing the wakeup. So if it was needed, someone would have noticed this problem. The symptom is _very_ hard to miss, as the whole server gets stuck hard, unable to do anything. It really is not acceptable to have a mutex implementation that can miss wakeups. This can stall the entire server. Even if the error monitor resolves the stall after 1 second, that is still unacceptable for many applications. Hope this helps, - Kristian.

2 3

[Maria-developers] Review: [MDEV-7062] parser refactoring: don't store field properties in LEX
by Alexander Barkov 16 Nov '14

16 Nov '14

Hi Sergei, > Sergei Golubchik edited comment on MDEV-7062 at 11/10/14 11:40 AM: > ------------------------------------------------------------------- > > http://lists.askmonty.org/pipermail/commits/2014-November/006908.html > > {noformat} > From: serg(a)mariadb.org > To: commits(a)mariadb.org > Subject: [Commits] 2c904b9: parser cleanup: don't store field properties > in LEX, use Create_field directly > Date: Mon, 10 Nov 2014 10:26:48 +0100 > {noformat} > Please find my comments below: > > [Commits] 2c904b9: parser cleanup: don't store field properties in LEX, use Create_field directly > serg at mariadb.org serg at mariadb.org > Mon Nov 10 11:26:48 EET 2014 > > Previous message: [Commits] Rev 4344: MDEV-6179: dynamic columns functions/cast()/convert() doesn't play nice with CREATE/ALTER TABLE in lp:~maria-captains/maria/5.5 > Next message: [Commits] Rev 4345: MDEV-6862 "#error <my_config.h>" and third-party libraries in lp:~maria-captains/maria/5.5 > Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] > > revision-id: 2c904b937b5ef7fbec60ebb9cb514a501c5bd8e4 > parent(s): f59588b9112965849986c45fcd150d9abed74063 > committer: Sergei Golubchik > branch nick: maria > timestamp: 2014-11-09 22:33:17 +0100 > message: > > parser cleanup: don't store field properties in LEX, use Create_field directly > > length/dec/charset are still in LEX, because they're also used > for CAST and dynamic columns. > > also > 1. fix "MDEV-7041 COLLATION(CAST('a' AS CHAR BINARY)) returns a wrong result" > 2. allow BINARY modifier in stored function RETURN clause > 3. allow "COLLATION without CHARSET" in SP/SF (parameters, RETURN, DECLARE) > 4. print correct variable name in error messages for stored routine parameters A very nice clean up. I only found it a little bit not easy to track that all Lex and Create_field attributes get initialized and copied to each other properly. I suggest to go a little bit further and get rid from Lex->dec, Lex->length and Lex->charset at all, and write the attributes directly to the destination as soon as they get parsed. See a proposed addon patch attached. > diff --git a/mysql-test/r/cast.result b/mysql-test/r/cast.result > index c81af13..8ad4568 100644 > --- a/mysql-test/r/cast.result > +++ b/mysql-test/r/cast.result > @@ -796,3 +796,32 @@ DATE("foo") > NULL > Warnings: > Warning 1292 Incorrect datetime value: 'foo' > +create table t1 (a int, b char(5) as (cast("a" as char(10) binary) + a) ); > +show create table t1; > +Table Create Table > +t1 CREATE TABLE `t1` ( > + `a` int(11) DEFAULT NULL, > + `b` char(5) AS (cast("a" as char(10) binary) + a) VIRTUAL > +) ENGINE=MyISAM DEFAULT CHARSET=latin1 > +drop table t1; > +select collation(cast("a" as char(10) binary)); > +collation(cast("a" as char(10) binary)) > +latin1_bin > +select collation(cast("a" as char(10) charset utf8 binary)); > +collation(cast("a" as char(10) charset utf8 binary)) > +utf8_bin > +select collation(cast("a" as char(10) ascii binary)); > +collation(cast("a" as char(10) ascii binary)) > +latin1_bin > +select collation(cast("a" as char(10) unicode binary)); > +collation(cast("a" as char(10) unicode binary)) > +ucs2_bin UNICODE is an alias for UCS2, which is not always compiled. Please move ucs2 related tests to ctype_ucs.test. <skip> > diff --git a/mysql-test/t/cast.test b/mysql-test/t/cast.test > index b6c37ca..aa43ee6 100644 > --- a/mysql-test/t/cast.test > +++ b/mysql-test/t/cast.test > @@ -456,3 +456,21 @@ SELECT CAST(TIME('10:20:30') AS DATE) + INTERVAL 1 DAY; > SET SQL_MODE=ALLOW_INVALID_DATES; > SELECT DATE("foo"); > > +# > +# CAST and field definition using same fields in LEX > +# > +create table t1 (a int, b char(5) as (cast("a" as char(10) binary) + a) ); > +show create table t1; > +drop table t1; > + > +# > +# CAST (... BINARY) > +# > +select collation(cast("a" as char(10) binary)); > +select collation(cast("a" as char(10) charset utf8 binary)); > +select collation(cast("a" as char(10) ascii binary)); > +select collation(cast("a" as char(10) unicode binary)); > +select collation(cast("a" as char(10) binary charset utf8)); > +select collation(cast("a" as char(10) binary ascii)); > +select collation(cast("a" as char(10) binary unicode)); More UCS2. You removed this code in field.cc: > - if (fld_length != NULL) > - { > - errno= 0; > - length= strtoul(fld_length, NULL, 10); > - if ((errno != 0) || (length > MAX_FIELD_BLOBLENGTH)) > - { > - my_error(ER_TOO_BIG_DISPLAYWIDTH, MYF(0), fld_name, MAX_FIELD_BLOBLENGTH); > - DBUG_RETURN(TRUE); > - } > - > - if (length == 0) > - fld_length= NULL; /* purecov: inspected */ > - } and added this in sql_yacc.yy: > + if (length) > + { > + int err; > + last_field->length= (ulong)my_strtoll10(length, NULL, &err); > + if (err) > + last_field->length= ~0UL; // safety > + } > + else > + last_field->length= 0; I have a feeling that on a 32bit platforms it will stop sending errors on attempt to create a blob larger that 4Gb. last_field->length is ulong, which is 32bit is a 32 bit platform, and it gets assigned to a 64 bit value.

2 2

[Maria-developers] Please review: sql_yacc.yy and HA_CREATE_INFO refactoring (pre-requisite for MDEV-5359 CREATE OR REPLACE, CREATE IF NOT EXISTS, DROP IF EXISTS)
by Alexander Barkov 14 Nov '14

14 Nov '14

Hi Sergei, The patch is attached. Thanks.

1 1

Re: [Maria-developers] [JIRA] (MDEV-7076) Build fails around libgroonga on some platforms in buildbot
by Alexander Barkov 12 Nov '14

12 Nov '14

Hello Kentoku, Mroonga fails to compile on some platforms. Can you please have a look? Thanks. On 11/12/2014 04:39 PM, Alexander Barkov (JIRA) wrote: > > [ https://mariadb.atlassian.net/browse/MDEV-7076?page=com.atlassian.jira.plug… ] > > Alexander Barkov updated MDEV-7076: > ----------------------------------- > Assignee: Kentoku (was: Alexander Barkov) > >> Build fails around libgroonga on some platforms in buildbot >> ----------------------------------------------------------- >> >> Key: MDEV-7076 >> URL: https://mariadb.atlassian.net/browse/MDEV-7076 >> Project: MariaDB Server >> Issue Type: Bug >> Components: OTHER >> Affects Versions: 10.0 >> Reporter: Elena Stepanova >> Assignee: Kentoku >> Priority: Critical >> Labels: buildbot, compile >> Fix For: 10.0.15 >> >> >> http://buildbot.askmonty.org/buildbot/builders/kvm-bintar-centos5-amd64/bui… >> http://buildbot.askmonty.org/buildbot/builders/kvm-deb-debian6-amd64/builds… >> http://buildbot.askmonty.org/buildbot/builders/kvm-deb-lucid-amd64/builds/4… >> It looks approximately like this, with some variations >> {noformat} >> standard input}: Assembler messages: >> {standard input}:28252: Error: open CFI at the end of file; missing .cfi_endproc directive >> gcc-4.4.real: Internal error: Killed (program cc1) >> Please submit a full bug report. >> See <file:///usr/share/doc/gcc-4.4/README.Bugs> for instructions. >> make[3]: *** [storage/mroonga/vendor/groonga/lib/CMakeFiles/libgroonga.dir/expr.c.o] Error 1 >> make[3]: *** Waiting for unfinished jobs.... >> {noformat} >> Given the list of platforms, I guess they all feature pretty old compilers, maybe that's the distinction. > > > > -- > This message was sent by Atlassian JIRA > (v6.4-OD-09-008#64005) >

1 0

[Maria-developers] Mroonga: wrapper.version_56_or_later_performance_schema is failing on BuildBot
by Alexander Barkov 12 Nov '14

12 Nov '14

Hello Kentoku, This test is failing in the latest builds on BB: mroonga/wrapper.version_56_or_later_performance_schema http://buildbot.askmonty.org/buildbot/builders/bld-dan-release/builds/1793/… Can you please check it? Thanks.

2 1

[Maria-developers] PLEASE REVIEWA: pre-requisite patch for the data type plugins, also fixing a number of optimizer bugs
by Alexander Barkov 12 Nov '14

12 Nov '14

Hello, While working on pluggable data types, I noticed a few problems related to optimizer: > MDEV-6950 Bad results with joins comparing DATE/DATETIME and INT/DECIMAL/DOUBLE/ENUM/VARCHAR columns > MDEV-6971 Bad results with joins comparing TIME and DOUBLE/DECIMAL columns > MDEV-6978 Bad results with joins comparing case insensitive VARCHAR/ENUM/SET expression to a _bin ENUM column > MDEV-6982 LEFT JOIN table elimination is not always used when it could > MDEV-6989 BINARY and COLLATE xxx_bin comparisions are not used for optimization in some cases > MDEV-6990 GROUP_MIN_MAX optimization is not applied in some cases when it could > MDEV-6991 GROUP_MIN_MAX optimization is erroneously applied in some cases In some cases wrong result sets are returned. So it would be nice to have this fixed in 10.0. I made these changes as a standalone patch. Please review. I also noticed some other optimizer bugs but not sure how to fix them properly: > MDEV-6986 Bad results with join comparing INT and VARCHAR columns > MDEV-6969 Bad results with joins comparing DOUBLE to BIGINT/DECIMAL columns > MDEV-6993 Bad results with join comparing DECIMAL and ENUM/SET columns Igor or SergeyP can possibly fix them on top of my changes. Also, I found some other problems (not related to optimizer): > MDEV-6973 XOR aggregates argument collations > MDEV-7005 NULLIF does not work as documented Not sure which version to fix them in. Please suggest. Thanks.

2 2

Re: [Maria-developers] MDEV-6885 - debs do not build on PPC64 in lp:maria/5.5
by Sergei Golubchik 11 Nov '14

11 Nov '14

Hi, Sergey! On Oct 17, Sergey Vojtovich wrote: > At lp:maria/5.5 > > ------------------------------------------------------------ > revno: 4327 > revision-id: svoj(a)mariadb.org-20141017112032-b0p2qvvlbn1xrfec > parent: svoj(a)mariadb.org-20141017110850-5rlt8fm3blxv4wtb > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 5.5 > timestamp: Fri 2014-10-17 15:20:32 +0400 > message: > MDEV-6885 - debs do not build on PPC64 > > Include ha_oqgraph.so into deb package only if it was able to build. Sorry, cannot help much. It looks ok, but I've had so many problems with our debian packaging - all these fixes looks right, and still need twenty iterations to actually be right. So, try it in buildbot, push in some tree that builds deb packages. If it'll work in all deb builders - please, push into 5.5. Regards, Sergei

3 3

Re: [Maria-developers] [Commits] MDEV-60 Support for Spatial Reference systems for the GIS data.
by Sergei Golubchik 11 Nov '14

11 Nov '14

Hi, Alexey! First, please, please, *do not* send patches from Thunderbird. It took me a lot of time to fix it, the patch was badly garbled. May be broken formatting everywhere in the patch was also caused by this. Then, I didn't see any tests in the patch, not even syntax tests, but also not create field, store, read tests. Please add them. Now, the review: > diff --git a/sql/field.cc b/sql/field.cc > --- a/sql/field.cc > +++ b/sql/field.cc > @@ -7808,6 +7808,35 @@ uint Field_blob::is_equal(Create_field *new_field) > > #ifdef HAVE_SPATIAL > > +uint gis_field_options_image(uchar *buff, List<Create_field> &create_fields) > +{ > + uint image_size= 0; > + uint n_field= 0; > + List_iterator<Create_field> it(create_fields); > + Create_field *field; > + while ((field= it++)) > + { > + if (field->sql_type != MYSQL_TYPE_GEOMETRY) > + continue; > + if (buff) > + int2store(buff + image_size, ((uint16) n_field)); > + image_size+= 2; > + if (buff) > + int2store(buff + image_size, ((uint16) field->length)); > + image_size+= 2; > + if (buff) > + int2store(buff + image_size, ((uint16) field->decimals)); > + image_size+= 2; > + if (buff) > + int4store(buff + image_size, ((uint32) field->srid)); > + image_size+= 4; > + n_field++; > + } > + > + return image_size; > +} Hmm... So, you basically, put an array of properties in frm, under EXTRA2_GIS Two comments: 1. Why n_field, length and decimals? I thought you only add srid, other properties are already stored elsewhere, aren't they? 2. Really think about upgrades downgrades. The "extra" section in the frm doesn't handle them well, I know how these problems happen. Think, what happens when you add more properties here? What if you remove some? How new server will parse old frms, how old server will read new frms? > + > + > void Field_geom::sql_type(String &res) const > { > CHARSET_INFO *cs= &my_charset_latin1; > @@ -9181,7 +9210,7 @@ bool Create_field::init(THD *thd, char *fld_name, enum_field_types fld_type, > Item *fld_on_update_value, LEX_STRING *fld_comment, > char *fld_change, List<String> *fld_interval_list, > CHARSET_INFO *fld_charset, uint fld_geom_type, > - Virtual_column_info *fld_vcol_info, > + char *fld_srid, Virtual_column_info *fld_vcol_info, fix the formatting, please > engine_option_value *create_opt, bool check_exists) > { > uint sign_len, allowed_type_modifier= 0; > @@ -9232,6 +9261,7 @@ bool Create_field::init(THD *thd, char *fld_name, enum_field_types fld_type, > pack_length= key_length= 0; > charset= fld_charset; > geom_type= (Field::geometry_type) fld_geom_type; > + srid= fld_srid ? (uint)atoi(fld_srid) : 0; and here > interval_list.empty(); > > comment= *fld_comment; > @@ -9635,7 +9665,7 @@ Field *make_field(TABLE_SHARE *share, uchar *ptr, uint32 field_length, > uint pack_flag, > enum_field_types field_type, > CHARSET_INFO *field_charset, > - Field::geometry_type geom_type, > + Field::geometry_type geom_type, uint srid, and here, and everywhere, I won't comment on that anymore > Field::utype unireg_check, > TYPELIB *interval, > const char *field_name) > diff --git a/sql/sql_lex.h b/sql/sql_lex.h > --- a/sql/sql_lex.h > +++ b/sql/sql_lex.h > @@ -2358,6 +2358,7 @@ struct LEX: public Query_tables_list > char *backup_dir; /* For RESTORE/BACKUP */ > char* to_log; /* For PURGE MASTER LOGS TO */ > char* x509_subject,*x509_issuer,*ssl_cipher; > + char *srid; really? it can only be used for field declarations, I don't think there's a need to put it in LEX see below. > String *wild; /* Wildcard in SHOW {something} LIKE 'wild'*/ > sql_exchange *exchange; > select_result *result; > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -6359,6 +6360,16 @@ precision: > } > ; > > +srid_option: > + /* empty */ > + { Lex->srid= (char *)0; } > + | > + SRID_SYM EQ NUM > + { > + Lex->srid=$3.str; > + } return the value of srid here, don't store it in lex. Like $$ = $3.str; or even $$ = atoi($3.str); > + ; > + > field_options: > /* empty */ {} > | field_opt_list {} > diff --git a/sql/table.cc b/sql/table.cc > --- a/sql/table.cc > +++ b/sql/table.cc > @@ -1451,6 +1464,20 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, > #ifdef HAVE_SPATIAL > geom_type= (Field::geometry_type) strpos[14]; > charset= &my_charset_bin; > + if (gis_options && gis_options_len >= 10) > + { > + gis_length= uint2korr(gis_options+2); > + gis_decimals= uint2korr(gis_options+4); > + srid= uint4korr(gis_options+6); > + gis_options_len-= 10; > + gis_options+= 10; If you have the function to generate this data in field.cc, I'd put a small function to read it in field.cc too. Just to have all reading/writing code in the same place. > + } > + else > + { > + srid= 0; > + gis_length= 0; > + gis_decimals= 0; > + } > #else > goto err; > #endif > diff --git a/sql/unireg.h b/sql/unireg.h > --- a/sql/unireg.h > +++ b/sql/unireg.h > @@ -190,6 +190,7 @@ enum extra2_frm_value_type { > #define EXTRA2_ENGINE_IMPORTANT 128 > > EXTRA2_ENGINE_TABLEOPTS=128, > + EXTRA2_GIS=129, better make it 2, not 129 (below EXTRA2_ENGINE_IMPORTANT threshold), because you de facto ignore it anyway when HAVE_SPATIAL is not defined. > }; > > int rea_create_table(THD *thd, LEX_CUSTRING *frm, > diff --git a/unittest/mysys/CMakeLists.txt b/unittest/mysys/CMakeLists.txt > --- a/unittest/mysys/CMakeLists.txt > +++ b/unittest/mysys/CMakeLists.txt > @@ -17,7 +17,7 @@ MY_ADD_TESTS(bitmap base64 my_vsnprintf my_atomic my_rdtsc lf my_malloc > LINK_LIBRARIES mysys) > > MY_ADD_TESTS(ma_dyncol > - LINK_LIBRARIES mysqlclient) > + LINK_LIBRARIES mysqlclient stdc++) Why? > > IF(WIN32) > MY_ADD_TESTS(my_delete LINK_LIBRARIES mysys) > Regards, Sergei

2 2

[Maria-developers] Fwd: [debian-mysql] Bug#766996: libmariadb-client-lgpl-dev-compat: misses a mysql_config symlink
by Daniel Schepler 10 Nov '14

10 Nov '14

I recently got the bug report below for the MariaDB C client library (Debian package). The request to add a compatibility symlink from mysql_config to mariadb_config makes sense to me -- do you see any potential problems with adding it? -- Daniel Schepler ---------- Forwarded message ---------- From: Rene Engelhard <rene(a)debian.org> Date: Mon, Oct 27, 2014 at 6:48 AM Subject: [debian-mysql] Bug#766996: libmariadb-client-lgpl-dev-compat: misses a mysql_config symlink To: Debian Bug Tracking System <submit(a)bugs.debian.org> Package: libmariadb-client-lgpl-dev-compat Version: 2.0.0-1 Severity: important https://packages.debian.org/sid/libmariadb-client-lgpl-dev-compat says: "MariaDB Client Library for C, compatibility symlinks MariaDB is a fast, stable and true multi-user, multi-threaded SQL database server. SQL (Structured Query Language) is the most popular database query language in the world. The main goals of MariaDB are speed, robustness and ease of use. This package includes compatibility symlinks to allow sources expecting the MySQL client libraries to be built against the MariaDB Client Library for C." The full contents are: https://packages.debian.org/sid/amd64/libmariadb-client-lgpl-dev-compat/fil… so it won't work with packages expecting mysql_config to give them the flags -> it should also ship a mysql_config -> mariadb_config link. Regards, Rene _______________________________________________ pkg-mysql-maint mailing list pkg-mysql-maint(a)lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-mysql-maint

2 1

Re: [Maria-developers] [Commits] Rev 4343: MDEV-6179: dynamic columns functions/cast()/convert() doesn't play nice with CREATE/ALTER TABLE in lp:~maria-captains/maria/5.5
by Oleksandr Byelkin 10 Nov '14

10 Nov '14

Hi, Sergei! Thank you! The patch is OK. On 08.11.14 19:54, Sergei Golubchik wrote: > At lp:~maria-captains/maria/5.5 > > ------------------------------------------------------------ > revno: 4343 > revision-id: sergii(a)pisem.net-20141108185442-37i07fh0bvm2atjr > parent: sergii(a)pisem.net-20141102163302-8ukz7pi4mwsx89n0 > committer: Sergei Golubchik <sergii(a)pisem.net> > branch nick: 5.5 > timestamp: Sat 2014-11-08 19:54:42 +0100 > message: > MDEV-6179: dynamic columns functions/cast()/convert() doesn't play nice with CREATE/ALTER TABLE > > When parsing a field declaration, grab type information from LEX before it's overwritten > by further rules. Pass type information through the parser stack to the rule that needs it. > === modified file 'mysql-test/r/create.result' > --- a/mysql-test/r/create.result 2013-05-11 17:31:50 +0000 > +++ b/mysql-test/r/create.result 2014-11-08 18:54:42 +0000 > @@ -2430,3 +2430,50 @@ a b > 1 1 > unlock tables; > drop table t1,t2; > +# > +# MDEV-6179: dynamic columns functions/cast()/convert() doesn't > +# play nice with CREATE/ALTER TABLE > +# > +create table t1 ( > +color char(32) as (COLUMN_GET(dynamic_cols, 1 as char)) persistent, > +cl char(32) as (COLUMN_GET(COLUMN_ADD(COLUMN_CREATE(1 , 'blue' as char), 2, 'ttt'), i as char)) persistent, > +item_name varchar(32) primary key, -- A common attribute for all items > +i int, > +dynamic_cols blob -- Dynamic columns will be stored here > +); > +INSERT INTO t1(item_name, dynamic_cols, i) VALUES > +('MariaDB T-shirt', COLUMN_CREATE(1, 'blue', 2, 'XL'), 1); > +INSERT INTO t1(item_name, dynamic_cols, i) VALUES > +('Thinkpad Laptop', COLUMN_CREATE(1, 'black', 3, 500), 2); > +select item_name, color, cl from t1; > +item_name color cl > +MariaDB T-shirt blue blue > +Thinkpad Laptop black ttt > +show create table t1; > +Table Create Table > +t1 CREATE TABLE `t1` ( > + `color` char(32) AS (COLUMN_GET(dynamic_cols, 1 as char)) PERSISTENT, > + `cl` char(32) AS (COLUMN_GET(COLUMN_ADD(COLUMN_CREATE(1 , 'blue' as char), 2, 'ttt'), i as char)) PERSISTENT, > + `item_name` varchar(32) NOT NULL, > + `i` int(11) DEFAULT NULL, > + `dynamic_cols` blob, > + PRIMARY KEY (`item_name`) > +) ENGINE=MyISAM DEFAULT CHARSET=latin1 > +drop table t1; > +create table t1 ( > +n int, > +c char(32) as (convert(cast(n as char), char)) persistent > +); > +insert into t1(n) values (1),(2),(3); > +select * from t1; > +n c > +1 1 > +2 2 > +3 3 > +show create table t1; > +Table Create Table > +t1 CREATE TABLE `t1` ( > + `n` int(11) DEFAULT NULL, > + `c` char(32) AS (convert(cast(n as char), char)) PERSISTENT > +) ENGINE=MyISAM DEFAULT CHARSET=latin1 > +drop table t1; > > === modified file 'mysql-test/t/create.test' > --- a/mysql-test/t/create.test 2012-12-17 20:34:56 +0000 > +++ b/mysql-test/t/create.test 2014-11-08 18:54:42 +0000 > @@ -2021,3 +2021,35 @@ connection default; > select * from t1; > unlock tables; > drop table t1,t2; > + > +--echo # > +--echo # MDEV-6179: dynamic columns functions/cast()/convert() doesn't > +--echo # play nice with CREATE/ALTER TABLE > +--echo # > +create table t1 ( > + color char(32) as (COLUMN_GET(dynamic_cols, 1 as char)) persistent, > + cl char(32) as (COLUMN_GET(COLUMN_ADD(COLUMN_CREATE(1 , 'blue' as char), 2, 'ttt'), i as char)) persistent, > + item_name varchar(32) primary key, -- A common attribute for all items > + i int, > + dynamic_cols blob -- Dynamic columns will be stored here > +); > +INSERT INTO t1(item_name, dynamic_cols, i) VALUES > + ('MariaDB T-shirt', COLUMN_CREATE(1, 'blue', 2, 'XL'), 1); > +INSERT INTO t1(item_name, dynamic_cols, i) VALUES > + ('Thinkpad Laptop', COLUMN_CREATE(1, 'black', 3, 500), 2); > + > +select item_name, color, cl from t1; > +show create table t1; > + > +drop table t1; > + > +create table t1 ( > + n int, > + c char(32) as (convert(cast(n as char), char)) persistent > +); > +insert into t1(n) values (1),(2),(3); > + > +select * from t1; > +show create table t1; > + > +drop table t1; > > === modified file 'sql/sql_lex.h' > --- a/sql/sql_lex.h 2014-08-02 19:26:16 +0000 > +++ b/sql/sql_lex.h 2014-11-08 18:54:42 +0000 > @@ -105,6 +105,15 @@ struct sys_var_with_base > LEX_STRING base_name; > }; > > +struct LEX_TYPE > +{ > + enum enum_field_types type; > + char *length, *dec; > + CHARSET_INFO *charset; > + void set(int t, char *l, char *d, CHARSET_INFO *cs) > + { type= (enum_field_types)t; length= l; dec= d; charset= cs; } > +}; > + > #ifdef MYSQL_SERVER > /* > The following hack is needed because mysql_yacc.cc does not define > > === modified file 'sql/sql_yacc.yy' > --- a/sql/sql_yacc.yy 2014-08-02 19:26:16 +0000 > +++ b/sql/sql_yacc.yy 2014-11-08 18:54:42 +0000 > @@ -737,6 +737,7 @@ static bool add_create_index (LEX *lex, > LEX_STRING lex_str; > LEX_STRING *lex_str_ptr; > LEX_SYMBOL symbol; > + LEX_TYPE lex_type; > Table_ident *table; > char *simple_string; > Item *item; > @@ -1471,13 +1472,15 @@ bool my_yyoverflow(short **a, YYSTYPE ** > %type <string> > text_string opt_gconcat_separator > > +%type <lex_type> field_def > + > %type <num> > type type_with_opt_collate int_type real_type order_dir lock_option > udf_type if_exists opt_local opt_table_options table_options > table_option opt_if_not_exists opt_no_write_to_binlog > opt_temporary all_or_any opt_distinct > opt_ignore_leaves fulltext_options spatial_type union_option > - start_transaction_opts field_def > + start_transaction_opts > union_opt select_derived_init option_type2 > opt_natural_language_mode opt_query_expansion > opt_ev_status opt_ev_on_completion ev_on_completion opt_ev_comment > @@ -5483,11 +5486,11 @@ field_spec: > field_def > { > LEX *lex=Lex; > - if (add_field_to_list(lex->thd, &$1, (enum enum_field_types) $3, > - lex->length,lex->dec,lex->type, > + if (add_field_to_list(lex->thd, &$1, $3.type, > + $3.length, $3.dec, lex->type, > lex->default_value, lex->on_update_value, > &lex->comment, > - lex->change,&lex->interval_list,lex->charset, > + lex->change, &lex->interval_list, $3.charset, > lex->uint_geom_type, > lex->vcol_info, lex->option_list)) > MYSQL_YYABORT; > @@ -5495,13 +5498,15 @@ field_spec: > ; > > field_def: > - type opt_attribute {} > - | type opt_generated_always AS '(' virtual_column_func ')' > - vcol_opt_specifier > - vcol_opt_attribute > - { > - $$= (enum enum_field_types)MYSQL_TYPE_VIRTUAL; > - Lex->vcol_info->set_field_type((enum enum_field_types) $1); > + type opt_attribute > + { $$.set($1, Lex->length, Lex->dec, Lex->charset); } > + | type opt_generated_always AS > + { $<lex_type>$.set($1, Lex->length, Lex->dec, Lex->charset); } > + '(' virtual_column_func ')' vcol_opt_specifier vcol_opt_attribute > + { > + $$= $<lex_type>4; > + Lex->vcol_info->set_field_type($$.type); > + $$.type= (enum enum_field_types)MYSQL_TYPE_VIRTUAL; > } > ; > > @@ -6892,11 +6897,11 @@ alter_list_item: > { > LEX *lex=Lex; > if (add_field_to_list(lex->thd,&$3, > - (enum enum_field_types) $5, > - lex->length,lex->dec,lex->type, > + $5.type, > + $5.length, $5.dec, lex->type, > lex->default_value, lex->on_update_value, > &lex->comment, > - $3.str, &lex->interval_list, lex->charset, > + $3.str, &lex->interval_list, $5.charset, > lex->uint_geom_type, > lex->vcol_info, lex->option_list)) > MYSQL_YYABORT; > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

Re: [Maria-developers] MDEV-6179: dynamic columns functions/cast()/convert() doesn't play nice with CREATE/ALTER TABLE in file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6179/
by Sergei Golubchik 08 Nov '14

08 Nov '14

Hi, Sanja! On Nov 02, sanja(a)askmonty.org wrote: > At file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6179/ > > ------------------------------------------------------------ > revno: 4341 > revision-id: sanja(a)askmonty.org-20141102133114-v46li6r2goyzhhp7 > parent: sanja(a)askmonty.org-20141031130729-dya4hci39x6zrv89 > committer: sanja(a)askmonty.org > branch nick: work-maria-5.5-MDEV-6179 > timestamp: Sun 2014-11-02 14:31:14 +0100 > message: > MDEV-6179: dynamic columns functions/cast()/convert() doesn't play > nice with CREATE/ALTER TABLE > > Type attributes (length, dec, charset) which parsed to global > variables now saved and restored during prsing types which can > appeared in a expression. Why did you create a stack in LEX, when bison generated parser already has a stack of states? Also, there's no need to store that in LEX, it doesn't need to live longer than the field_spec rule. By coincidence I've spent a couple of last days rewriting this very part of the parser. I've also fixed this bug without knowing it :) But in 10.1, and the patch is too big to put it in 10.0, let alone 5.5, so it's useless here. Still, I tried to create a smaller patch for this bug in 5.5. Please, review, it's on the commit list. Regards, Sergei

1 0

Re: [Maria-developers] MariaDB 10.0.14 got rid of replication safety warning without warning in release notes?
by Pavel Ivanov 08 Nov '14

08 Nov '14

And here's response from Monty to my last email. And this response together with the whole thread makes me really worried. It alludes to the fact that all MySQL and MariaDB developers before making any changes should read the whole codebase first and then make sure that their changes work the same way and that they conform to the intentions of original code developers even though there's no way they can talk to those original developers. AFAIK that's not how development on big projects with hundreds of participants works. And it really raises the question for me: what kind of quality the product that is developed with such principles in mind can have? Sergei Golubchik, I believe you are the technical leader of MariaDB project. Could you please tell me what you think about this issue? On Tue, Nov 4, 2014 at 6:32 AM, Michael Widenius <michael.widenius(a)gmail.com> wrote: > > Hi! > >>>>>> "Pavel" == Pavel Ivanov <pivanof(a)google.com> writes: > > Pavel> On Sun, Nov 2, 2014 at 3:27 PM, Michael Widenius <monty(a)mariadb.org> wrote: >>>>> This is the same order that keys are presented in SHOW CREATE TABLE >>> > Pavel> Why is it safe to assume that keys have the same order on master and on slave? >>> >>> Primary and Unique keys should always be ordered the same way on >>> master and slave. This is handled by the 'sort_keys()' function that >>> you can find at: sql/sql_table.cc, line 2770. > > Pavel> This function seem to sort only by "type" of the key. If several > Pavel> unique keys have the same "type" according to its classification then > Pavel> they will be in the "original order" as the comment in the function > Pavel> says. And the original order may be different. > > This function ensures that the table will be identical on slave and > master as long as one of the following requirements are fulfilled: > > - You are using the same create table and alter table statements on > both machines. > - You are using the same CREATE TABLE statement as 'show create' shows > on the master. > - You are adding unique keys in the same order on master and slave. > > This is always the case when you are using the replication that MySQL > and MariaDB uses. > > If you are creating tables differently on the slave than on the > master, then you are on your own. > > So in all normal and most abnormal cases, the table will be created > identically on the master and slave, even if the slave is using a > different storage engine, have different columns or have additional > keys than on the master. > > <cut> > >>> I added a test for this to: >>> mysql-test/t/insert.test >>> >>> --echo # >>> --echo # MDEV-5168: Ensure that we can disable duplicate key warnings >>> --echo # from INSERT IGNORE >>> --echo # > > Pavel> This test doesn't check which duplicate key is asserted to execute ON > Pavel> DUPLICATE KEY action. > > This test verifies that we don't get warnings for the tests. > Elinas test verifies which key is used. > >>> Elena added another test for this in >>> mysql-test/suite/storage_engine/insert_with_keys.test >>> >>> See line 142, that start with: >>> --let $create_definition = a $int_indexed_col UNIQUE KEY, b $int_indexed_col UNIQUE KEY, c $int_col > > Pavel> This test seem to check that UPDATE is executed on the correct > Pavel> duplicate key, but it's not executed during regular mtr run. How and > Pavel> when it's executed in MariaDB? > > The storage engine tests are to be used by people that implements > their own storage engine for MariaDB. > > We run the extended tests from time to time to verify storage engines. > We will also run it on some buildbot servers when running releae builds. > The storage engine tests is not something that has to be run for > normal builds. > >>> What is important is that all storage engine should work the same >>> way. If one returns something different compared to any of the >>> existing engines then it's a bug. >>> >>> In this case the implicit documentation is how MyISAM, Aria and InnoDB >>> works. All of them returns the first conflicting key. > > Pavel> I'm sorry to say, but you've explained exactly the reason why this > Pavel> behavior is not guaranteed and can be changed at any time. > > It can only be changed if someone adds new code that breaks the > storage engine interface, as defined by existing engines and one > doesn't check it with the storage engine test suite. > > Pavel> Yes, I > Pavel> understand that when you wrote the interface you had some particular > Pavel> requirements in mind. But ultimately it's up to people actively > Pavel> working on the code (who I believe are counted by hundreds now) to > Pavel> decide whether to maintain the particular behavior or change > it. > > It's the MariaDB captains that are responsible to maintain the > interface. The only way one can change something is to propose a > change and get it accepted. > > It's not free for anyone to just change old functionality and expect > that to be accepted in the standard version. > > In all cases, including this, it's an absolute requirement that > engines MUST work exactly as existing engines, so that a user can > switch engines and expect them to work identically. > > (The only exceptions are things that are impossible to fix, like > the differences between transactional and not transactional tables and > that there is no defined row order in tables). > > If engine writes don't ensure this, then you will get data corruption > on the server by just running with different engines. > > For case on insert on duplicate key, there is no acceptable reason > for engines to work differently. > > Pavel> And > Pavel> as nobody knows the original requirements. > > The original requirements was well defined. Just that non one bother > to ask doesn't mean that the requirement didn't exist. > > And the fact that all existing engines in MariaDB ensures this shows > that the requirements has been understood. > > In the case when we notice that two engines works differently, it's up > to the MariaDB captains to fix this! > The fix is NOT to extend the API and allow things to work differently. > > > Pavel> and the reasons why they > Pavel> can't be changed, and when the behavior is not tested anywhere > > We add new tests when we notice that they are needed, as we did in > this case. > For example, the case with INSERT ... ON DUPLICATE KEY is now tested. > > Pavel> and > Pavel> doesn't have any documentation even in the weakest form of comments or > Pavel> asserts, the decision is usually very easy -- change at will, nobody > Pavel> should ever depend on implementation details. > > Yes, no one should depend in implementation details, except when the > implementation detail is different from other engines and it results > in different data. > > If that is the case then the engine that does things differently than > the existing engines must be changed. > > Pavel> Even if you promise to > Pavel> fix storage engines that change this behavior, by the time you notice > Pavel> the change you'll likely won't be able to fix it (as e.g. in case of > Pavel> https://mariadb.atlassian.net/browse/MDEV-6916) or someone will > Pavel> already experience a data corruption. > > In most cases we can fix this. What was unfortunate with the views > code was that we didn't store the MariaDB version number in the view > definition. This is to be fixed in 10.1 so that this will never happen > again! > > Pavel> I'd strongly recommend to revert this change. > > Sorry, but I can't agree to revert the change because: > > a) Something that can't ever happen in MariaDB (when used correctly > with the including engines). > b) Allowing a behavior that could for the same command result in > different data sets, depending on the storage engine. > c) I have not seen a real world > > One of my main jobs is to ensure that the storage engine API is > evolved in a resonable manner and if there is a conflict between how > storage engine works ensure that things are resolved so that the user > can expect the same results independent on the storage engine. > > Reverting the change would go directly agains the above and also > against what I belive is the future of the storage engine API. > > Regards, > Monty

2 1

Re: [Maria-developers] MariaDB 10.0.14 got rid of replication safety warning without warning in release notes?
by Pavel Ivanov 08 Nov '14

08 Nov '14

Monty, can you please subscribe all your email address to maria-develoeprs@ ? People reading the list cannot see your emails. I'm forwarding here the unaltered message from Monty that I was answering to in my last email. On Sun, Nov 2, 2014 at 3:27 PM, Michael Widenius <monty(a)mariadb.org> wrote: > > Hi! > > I have again to apologize for a late reply. > After my last reply to your topic I traveled to China and noticed > that the firewall there has got much better; You can't access google, > twitter, facebook, blogspot etc :( I tried to connect over a VPN, but > that didn't work for me either. > > It did feel a bit strange to not be able to do computer work for more > than a week, but the lovely people in China who took care of me did > compensate for that. > >>>>>> "Pavel" == Pavel Ivanov <pivanof(a)google.com> writes: > > Pavel> Which key is "the first duplicate key". >>> >>> The first key that has a duplicate value in the order of keys in the KEY >>> structure that MariaDB provides for the storage engine. > > Pavel> I'm definitely not that familiar with the relevant MariaDB code, but I > Pavel> didn't find KEY structure to be passed to any insert-related function. > Pavel> Can you elaborate (preferably with code references) how this > Pavel> restriction is supposed to work? > > The following code examples are from 10.0. > > All handler functions get access to the TABLE object, which defines > the record format and the keys. > > The keys are stored in TABLE->key_info (see table.h, line 1062) > > When one reads a key, one refers to keys according to the number in > the key_info structure. For example: > > A set of reads by key is done by calling: > > handler->ha_index_init(key_number,..); > read_until_no_more_rows > handler->ha_index_read_map(). > handler->ha->index_end() > > When one calls handler->info() to ask for a which key caused a > duplicate error, it's supposed to store the first duplicate key (key > with smallest number) in handler->errkey. > >>> This is the same order that keys are presented in SHOW CREATE TABLE > > Pavel> Why is it safe to assume that keys have the same order on master and on slave? > > Primary and Unique keys should always be ordered the same way on > master and slave. This is handled by the 'sort_keys()' function that > you can find at: sql/sql_table.cc, line 2770. > > The only case when this is not the case, is if you add an unique key > on the slave that was not on the master and there is conflicts when > you insert things because of this key. > > However, if you do this, I would say that you are on your own. > >>> To help engine writes we are constantly adding new test in the storage >>> engine test suite, to make it easier for storage engine vendors to >>> test that their storage engine follows the interface. >>> We have now added a test to check for the correct behaviour for the >>> case of INSERT ... ON DUPLICATE KEY. > > Pavel> This is very interesting. Can you say which test it is and what > Pavel> exactly does it test? Is it present in 10.0.14 or it's only at the > Pavel> latest 10.0 branch? > > I added a test for this to: > mysql-test/t/insert.test > > --echo # > --echo # MDEV-5168: Ensure that we can disable duplicate key warnings > --echo # from INSERT IGNORE > --echo # > > Elena added another test for this in > mysql-test/suite/storage_engine/insert_with_keys.test > > See line 142, that start with: > --let $create_definition = a $int_indexed_col UNIQUE KEY, b $int_indexed_col UNIQUE KEY, c $int_col > >>> Do you have or use a storage engine that doesn't provide the expected value >>> for the conflicting key ? > > Pavel> I don't have a specific example of storage engine that detects > Pavel> duplicate key in different order, but I'm not convinced and don't have > Pavel> hard proof that such engine cannot exist and that InnoDB itself is not > Pavel> susceptible to this problem. I can easily imagine though that even > Pavel> given the particular order of keys in table definition the key > Pavel> reported as having duplicate can depend on the way storage engine > Pavel> stores those keys. > > As key order is defined on the upper level, above InnoDB, this should > be 100 % safe (at least for InnoDB). > > For other store engines, this is safe for any engine that test for > duplicate key in key order. > > If we ever find a case where it's not safe with any storage engine > that we deliver with MariaDB, we will fix the storage engine. > > It's ok if they do thing differently so that they can detect duplicate > key for other keys faster. > > However, it has been required from all storage engines from start that > if one calls 'handler->info()' to request which key was duplicated, > they must return the first key. > > If not, they are breaking the storage engine interface (both for MySQL > and MariaDB) as there is no way the upper level can guarantee that > things will work identically between storage engines. > > Pavel> BTW, can you point to MySQL documentation and/or comment in the code > Pavel> that indicates this requirement to storage engine implementation? > > Most of the storage engine interface is not 100 % documented. This is > one of the case. > However, I wrote the code related to errkey and this is how it was > supposed to work. > In general if something is not documented, then it's up to the people > that created the interface or are actively working on it to define how > it should work. > > What is important is that all storage engine should work the same > way. If one returns something different compared to any of the > existing engines then it's a bug. > > In this case the implicit documentation is how MyISAM, Aria and InnoDB > works. All of them returns the first conflicting key. > > Pavel> On the related note Release Notes mention the error log flood > Pavel> protection. Why it's not behind flag? How should I disable the feature > Pavel> and get back to the old behavior? > >>> We already had in MariaDB flood protection for some error >>> messages. > > Pavel> Can you say exactly which error messages are suppressed already? Is > Pavel> there a documentation link about that? > > All 'unsafe logging' errors are suppressed. > See enum enum_binlog_stmt_unsafe in sql_lex.h > (This is what I know). > >>> I am happy to look into adding a flag that is usable for any kind of >>> suppressed messages. >>> >>> A couple of ways to do this: >>> >>> a) Not suppress anything if you give --log-warnings a value > 2 >>> b) make LIMIT_UNSAFE_WARNING_ACTIVATION_TIMEOUT a configurable >>> variable, where 0 would mean no suppression. >>> >>> Which of the above, or both, would you prefer? >>> Or do you have any other suggestion of how to do this? > > Pavel> Reading documentation on --log-warnings I don't see how it can be > Pavel> related to the error suppression. So I think making > Pavel> LIMIT_UNSAFE_WARNING_ACTIVATION_TIMEOUT configurable would be much > Pavel> better, though it should apply to all possible error suppressions. > > Currently we only have suppression for a limited set of 'unsafe' > errors, but I agree that when we add a flag we should name this with > a general name to be able to reuse it for all possible future > suppressions. > > In 10.1 we are looking at adding a feature that one can suppress any > error from the error log, but that's a different topic. > > Regards, > Monty

1 0

[Maria-developers] Maybe a new information schema table for replication?
by Roberto Spadim 06 Nov '14

06 Nov '14

Hi guys, i got one question these days and i don't know how to solve... the question is: "How to see the queries executing by replication thread?" i'm thinking about a information schema plugin to see whats is 'queued' in replication thread, at "master" (write to binlog) and "slave" (executing from binlog) side any idea if this is possible? i didn't checked or read any part of binlog/replication code yet thanks guys! :) -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

[Maria-developers] pcre.h should be installed if MariaDB is built with bundled pcre ?
by Kazuhiko Shiozaki 06 Nov '14

06 Nov '14

Hi, pcre.h is not installed by 'make install' even if MariaDB is built with bundled pcre. But pcre.h is referred in one of installed header files like : include/mysql/private/item_cmpfunc.h:#include "pcre.h" /* pcre header file */ So I think that it is better to install pcre.h as well if MariaDB is built with bundled pcre. What do you think ? -- Kazuhiko Shiozaki, Nexedi SA Senior Consultant Nexedi: Consulting and Development of Free / Open Source Software http://www.nexedi.com/ ERP5: Full Featured High End Open Source ERP http://www.erp5.com/

2 1

[Maria-developers] Fwd: Is it soon 'end of this year'?
by Peter Laursen 04 Nov '14

04 Nov '14

Forwarding to MariaDB developers mailing list. It may be a matter for somebody else than Sergei (Georg, for instance) (and "here" means"Webyog"). Abishek detailed errors in a chat with me "symbols like Snippet mysql_client_find_plugin , Snippet mysql_client_register_plugin are not found" ..exactly like in the MySQL bug report. Do you need a JIRA report with a full test case? -- Peter ---------- Forwarded message ---------- From: Abhishek Kumar Pandey <abhishek.pandey(a)webyog.com> Date: Tue, Nov 4, 2014 at 12:51 PM Subject: Re: [Maria-developers] Is it soon 'end of this year'? To: Peter Laursen <peter_laursen(a)webyog.com> Cc: "team. sqlyog" <team.sqlyog(a)webyog.com>, serg(a)mariadb.org, "team.monyog" <team.monyog(a)webyog.com> I am the developer here working with this. I am facing same problem as reported here- http://bugs.mysql.com/bug.php?id=69204 <http://www.google.com/url?q=http%3A%2F%2Fbugs.mysql.com%2Fbug.php%3Fid%3D69…> . Is the same fix needed in the mariadb library? On Tue, Oct 14, 2014 at 6:58 PM, Peter Laursen <peter_laursen(a)webyog.com> wrote: > You will get to this page: https://edelivery.oracle.com/ > > I think you should use a non-webyog email address. Not sure what Mahesh > did. He downloaded frequently in order to check latest MEM. > > > --Peter > > On Tue, Oct 14, 2014 at 3:26 PM, Peter Laursen <peter_laursen(a)webyog.com> > wrote: > >> You could also try to add the code as described by Sergei. First step is >> if it will compile and connect to servers not usinglain text/LDAP. >> >> -- Peter >> >> On Tue, Oct 14, 2014 at 3:12 PM, Supratik Goswami < >> supratik.goswami(a)webyog.com> wrote: >> >>> On Tue, Oct 14, 2014 at 4:58 PM, Sudhindra Bhat < >>> sudhindra.bhat(a)webyog.com> wrote: >>> >>>> +supratik >>>> >>>> We will check this once we have access to MySQL Enterprise server from >>>> Oracle. >>>> >>>> @supratik, >>>> Do we have the binary for MySQL Enterprise server? A Trial is also >>>> sufficient. >>>> >>> >>> No, I think. You may download it. >>> >>>> >>>> On Tue, Oct 14, 2014 at 2:52 PM, Peter Laursen < >>>> peter_laursen(a)webyog.com> wrote: >>>> >>>>> We need to try this again. I don't rememebr who gave up supporting >>>>> 'clear text plugin' with MariaDB API back in April 2013. Probably it was >>>>> not Sudhindra, Likely Shubhansh or Vishal. @Vishal: do you remember >>>>> anything? >>>>> >>>>> I remember one detail from the old discussion we had with MariaDB. >>>>> They told something like "Once enabled nothing special need to be done to >>>>> support clear text. The client wll automatically detect when the server >>>>> requests the password sent as clear text". This mean as far as I can >>>>> understand that we will - unlke early 11.x version - not need a checkbox in >>>>> the connection manager. >>>>> >>>>> We also need access to a commercial server (TRIAL) from Oracle. Mahesh >>>>> could arrange that. Who can today? >>>>> >>>>> >>>>> -- Peter >>>>> >>>>> >>>>> ---------- Forwarded message ---------- >>>>> From: Sergei Golubchik <serg(a)mariadb.org> >>>>> Date: Mon, Oct 13, 2014 at 11:24 PM >>>>> Subject: Re: [Maria-developers] Is it soon 'end of this year'? >>>>> To: Peter Laursen <peter_laursen(a)webyog.com> >>>>> >>>>> >>>>> Hi, Peter! >>>>> >>>>> On Oct 13, Peter Laursen wrote: >>>>> > Thanks .. you probabaly know that I am not a developer rmyself (I >>>>> basically >>>>> > stopped being so when ALGOL died 35 years ago!). I have shared all >>>>> > information with SQLyog developer team and we will look into this >>>>> again. >>>>> >>>>> Sure! Don't hesitate to tell if something won't work for you. >>>>> >>>>> Regards, >>>>> Sergei >>>>> >>>>> >>>> >>>> >>>> -- >>>> Regards, >>>> Sudhindra Bhat >>>> >>> >>> >> >

2 1

Re: [Maria-developers] MariaDB 10.0.14 got rid of replication safety warning without warning in release notes?
by Pavel Ivanov 03 Nov '14

03 Nov '14

On Sun, Nov 2, 2014 at 3:27 PM, Michael Widenius <monty(a)mariadb.org> wrote: >>> This is the same order that keys are presented in SHOW CREATE TABLE > > Pavel> Why is it safe to assume that keys have the same order on master and on slave? > > Primary and Unique keys should always be ordered the same way on > master and slave. This is handled by the 'sort_keys()' function that > you can find at: sql/sql_table.cc, line 2770. This function seem to sort only by "type" of the key. If several unique keys have the same "type" according to its classification then they will be in the "original order" as the comment in the function says. And the original order may be different. > The only case when this is not the case, is if you add an unique key > on the slave that was not on the master and there is conflicts when > you insert things because of this key. > > However, if you do this, I would say that you are on your own. > >>> To help engine writes we are constantly adding new test in the storage >>> engine test suite, to make it easier for storage engine vendors to >>> test that their storage engine follows the interface. >>> We have now added a test to check for the correct behaviour for the >>> case of INSERT ... ON DUPLICATE KEY. > > Pavel> This is very interesting. Can you say which test it is and what > Pavel> exactly does it test? Is it present in 10.0.14 or it's only at the > Pavel> latest 10.0 branch? > > I added a test for this to: > mysql-test/t/insert.test > > --echo # > --echo # MDEV-5168: Ensure that we can disable duplicate key warnings > --echo # from INSERT IGNORE > --echo # This test doesn't check which duplicate key is asserted to execute ON DUPLICATE KEY action. > Elena added another test for this in > mysql-test/suite/storage_engine/insert_with_keys.test > > See line 142, that start with: > --let $create_definition = a $int_indexed_col UNIQUE KEY, b $int_indexed_col UNIQUE KEY, c $int_col This test seem to check that UPDATE is executed on the correct duplicate key, but it's not executed during regular mtr run. How and when it's executed in MariaDB? >>> Do you have or use a storage engine that doesn't provide the expected value >>> for the conflicting key ? > > Pavel> I don't have a specific example of storage engine that detects > Pavel> duplicate key in different order, but I'm not convinced and don't have > Pavel> hard proof that such engine cannot exist and that InnoDB itself is not > Pavel> susceptible to this problem. I can easily imagine though that even > Pavel> given the particular order of keys in table definition the key > Pavel> reported as having duplicate can depend on the way storage engine > Pavel> stores those keys. > > As key order is defined on the upper level, above InnoDB, this should > be 100 % safe (at least for InnoDB). > > For other store engines, this is safe for any engine that test for > duplicate key in key order. > > If we ever find a case where it's not safe with any storage engine > that we deliver with MariaDB, we will fix the storage engine. > > It's ok if they do thing differently so that they can detect duplicate > key for other keys faster. > > However, it has been required from all storage engines from start that > if one calls 'handler->info()' to request which key was duplicated, > they must return the first key. > > If not, they are breaking the storage engine interface (both for MySQL > and MariaDB) as there is no way the upper level can guarantee that > things will work identically between storage engines. > > Pavel> BTW, can you point to MySQL documentation and/or comment in the code > Pavel> that indicates this requirement to storage engine implementation? > > Most of the storage engine interface is not 100 % documented. This is > one of the case. > However, I wrote the code related to errkey and this is how it was > supposed to work. > In general if something is not documented, then it's up to the people > that created the interface or are actively working on it to define how > it should work. > > What is important is that all storage engine should work the same > way. If one returns something different compared to any of the > existing engines then it's a bug. > > In this case the implicit documentation is how MyISAM, Aria and InnoDB > works. All of them returns the first conflicting key. I'm sorry to say, but you've explained exactly the reason why this behavior is not guaranteed and can be changed at any time. Yes, I understand that when you wrote the interface you had some particular requirements in mind. But ultimately it's up to people actively working on the code (who I believe are counted by hundreds now) to decide whether to maintain the particular behavior or change it. And as nobody knows the original requirements and the reasons why they can't be changed, and when the behavior is not tested anywhere and doesn't have any documentation even in the weakest form of comments or asserts, the decision is usually very easy -- change at will, nobody should ever depend on implementation details. Even if you promise to fix storage engines that change this behavior, by the time you notice the change you'll likely won't be able to fix it (as e.g. in case of https://mariadb.atlassian.net/browse/MDEV-6916) or someone will already experience a data corruption. I'd strongly recommend to revert this change. Thank you, Pavel

1 0

[Maria-developers] MDEV-5528 Command line variable to choose MariaDB-5.3 vs MySQL-5.6 temporal data formats
by Alexander Barkov 03 Nov '14

03 Nov '14

Hello Ian, I've pushed MDEV-5528 into 10.1. I guess it will need some documenting. Can you please have a look? Thanks.

1 0

[Maria-developers] nginx sources in mroonga
by Alexander Barkov 03 Nov '14

03 Nov '14

Hi Kentoku, Sergei Golubchik is asking why we have complete nginx sources inside mroonga? Can you remove them please? Thanks.

3 2

[Maria-developers] Cross-compiling MariaDB galera cluster
by Sylvain Raybaud 30 Oct '14

30 Oct '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dear MariaDB community, Recently I've been trying to cross-compile MariaDB and I've been running in some problems. I'd like to have your opinion on them. 1) Determining stack direction 2) Building Percona XtraDB 3) Building jemalloc Let me first tell that I'm trying to build MariaDB galera version 10.0.12. I'm compiling for Arm but may be compiling for something else in the unpredictable future. 1) Determining stack direction I started by passing the following options to cmake (via the buildroot system): - -DCMAKE_CROSSCOMPILING=1 -DWITH_WSREP=1 -DWITH_INNODB_DISALLOW_WRITES=1 The first problem I ran into was cmake complaining: STACK_DIRECTION is not defined. Please specify -DSTACK_DIRECTION=1 or -DSTACK_DIRECTION=-1 when calling cmake. It seems that STACK_DIRECTION is not automatically determined when cross-compiling. I may determine the correct value for a given target. But if I need a portable solution (and I need one in order to create a buildroot package so that I can build for many different target without too much hassle) I'm not sure what my options are. It seems that the toolchain is aware of this value since stack direction (or more precisely STACK_GROWS_DOWNWARD) is set in ./output/host/opt/ext-toolchain/lib/gcc/arm-none-linux-gnueabi/4.8.1/plugin/include/config/arm/arm.h Maybe there is a way to use this somehow? What do you think? 2) Building Percona XtraDB For the sake of trying I explicitly set -DSTACK_DIRECTION=1 just to see what happens. Building goes on until it reaches the xtradb module. Then it fails with: CMake Error at storage/xtradb/CMakeLists.txt:434 (MESSAGE): Percona XtraDB is not supported on this platform (Here I'll just copy-paste something I wrote on maria-discuss list a few days ago) This error is triggered in storage/xtradb/CMakeLists.txt when XTRADB_OK is not set; it seems to me that XTRADB_OK is set in either one of the following cases: a) CMAKE_CROSSCOMPILING is NOT set and some tests are successful (thus setting HAVE_IB_GCC_ATOMIC_BUILTINS) or b) We're compiling for MSVC or CMAKE_SYSTEM_NAME is "SunOS" and some other conditions are met. Since I'm not compiling for MSVC or SunOS and I'm crosscompiling indeed, XTRADB_OK is not set and the error is triggered. Here I'm stuck. I've got the feeling that as such one cannot cross-compile xtradb because XTRADB_OK will never be set; or maybe with proper options passed to cmake the tests would be made elsewhere and HAVE_IB_GCC_ATOMIC_BUILTINS would be set beforehand? Or maybe I should patch this CMakeLists.txt? Or maybe I should just compile without xtradb altogether, if that's possible? Finally I just passed -DXTRADB_OK just to see what would happen, again (obviously there is little chance that it would result in anything usable). And reached problem #3: 3) Building jemalloc This one fails at configure time with: checking whether we are cross compiling... configure: error: in `/home/sylvain/DEPOTS/buildroot/output/build/mariadb-galera-10.0.12/extra/jemalloc/build': configure: error: cannot run C compiled programs. If you meant to cross compile, use `--host'. Somehow I was expecting that having passed -DCMAKE_CROSSCOMPILING=1 and the host being set by buildroot it would propagate down to the modules... Am I missing something? At this point I'm out of dirty hacks, even to "see what happens" ;) Cheers, - -- Sylvain Raybaud www.green-communications.fr -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJT4ke/AAoJEEkkwl4JtJ9yJW0QALBz3wj0VsfMtUBUabP13q74 ++y1AiCLZ0js+UGCqxB9PVz4yvhhnMAz8QnPpH37XJtDm+VaJBQTEZL65CHF3nlr dHe46/q/wswlbdhNsQu6JmMe3JoOe2TFbjDenQMiUvW9mOilt86ioveHVLGYbpnq 5LYMrPlYXUBpDcxEYXXbTpIHsAoHUrOE1Gj5AsxwbUvQuHDDr9M/Ws01iIXLbzT8 0pImhgBPYUwqaj9tJnPkls+9LzoOb2m269n9TCbmWk0ixakpHjeNMn5lnRDhOwNh AKkHrGdMUomSBca4G0kQoeNxwC2rCjO8+bRxjh7v6GTdME/sgsgn3yOv2EznldUr tFaO/JfraHSUgmiRyVo41U3uTESxZKRJYOV9IqUhl4HiS85D5oa/bD6l84rhx95J 52/hUoApoStr8tSk0tzXk1/UWJEQ3fUZ7XoXqgWq+c1uMgbgZftAAqAw7Xvmc88t tf69FqBzw5dpf8Kg0I7iAggBpBLIWGJwnlVvcmLY4UUE1cDFfZstMBSXaS155LQF VClK+ytE2Ef8oAB9/nlMXgg4b3WXlD0PRQDRTNIwCAzdvsuu+S0q64xLtw/JYnLl KPcLWfwXy+kRhT+vtUWtqMjjkSQ3e6rJDybK11vgYIAWm/Xfk5s8i0g7Bgmnf3wp JGYUAR285gY0XMqL6LmU =vBo4 -----END PGP SIGNATURE-----

2 4

[Maria-developers] Valgrind failure in Buildbot
by Kristian Nielsen 30 Oct '14

30 Oct '14

Hi Sergey, We briefly discussed the new valgrind error in buildbot after your push on IRC: https://buildbot.askmonty.org/buildbot/builders/work-amd64-valgrind/builds/… ==13161== Conditional jump or move depends on uninitialised value(s) ==13161== at 0x4C280AA: memcpy (mc_replace_strmem.c:115) ==13161== by 0x723F6B: Field_varstring::get_key_image(unsigned char*, unsigned int, Field::imagetype) (field.cc:6932) ==13161== by 0x818D5B: key_copy(unsigned char*, unsigned char*, st_key*, unsigned int, bool) (key.cc:146) ==13161== by 0x6DD6FE: Mrr_ordered_index_reader::interrupt_read() (multi_range_read.cc:448) ==13161== by 0x6DE605: Mrr_ordered_rndpos_reader::refill_from_index_reader() (multi_range_read.cc:688) ==13161== by 0x6DE7D2: Mrr_ordered_rndpos_reader::refill_buffer(bool) (multi_range_read.cc:618) ==13161== by 0x6DFB27: DsMrr_impl::dsmrr_init(handler*, st_range_seq_if*, void*, unsigned int, unsigned int, st_handler_buffer*) (multi_range_read.cc:966) ==13161== by 0x91AAE9: ha_innobase::multi_range_read_init(st_range_seq_if*, void*, unsigned int, unsigned int, st_handler_buffer*) (ha_innodb.cc:13961) ==13161== by 0x6DA60A: JOIN_TAB_SCAN_MRR::open() (sql_join_cache.cc:3867) ==13161== by 0x6D7363: JOIN_CACHE::join_matching_records(bool) (sql_join_cache.cc:2249) ==13161== by 0x6D5647: JOIN_CACHE::join_records(bool) (sql_join_cache.cc:2086) ==13161== by 0x5F8259: sub_select_cache(JOIN*, st_join_table*, bool) (sql_select.cc:16717) ==13161== by 0x6D71BA: JOIN_CACHE::generate_full_extensions(unsigned char*) (sql_join_cache.cc:2397) ==13161== by 0x6D74E6: JOIN_CACHE::join_matching_records(bool) (sql_join_cache.cc:2289) ==13161== by 0x6D5647: JOIN_CACHE::join_records(bool) (sql_join_cache.cc:2086) ==13161== by 0x5F8219: sub_select_cache(JOIN*, st_join_table*, bool) (sql_select.cc:16697) I took a quick look. The problem seems to be that the code is copying out the fields of a key from table->record[0] (key_copy()), however, the first length byte of the value is uninitialised. I think it's the field col_varchar_key in the table, testcase is appended below. I checked a bit back up the callstack. The field col_varchar_key is reported as uninitialised already at the start of Mrr_ordered_rndpos_reader::refill_from_index_reader(), but I'm not really familiar with when and how table->record[0] is supposed to be initialised during multi-range read. As you said on IRC, your recent patch is related to interrupt_read(), so probably best if you take a look and see if you can figure out something. Incidentally, I noticed something that looks wrong in key_copy(to_key, from_record, key_info, key_length, with_zerofill) The comments say that the key is copied from `from_record', however actually the function only works if the from_record passed in is table->record[0] (or more precisely, the same record as is installed in key_info->key_part->field). Because it checks the null bit in from_record, but copies the data from table->record[0]. However, this is not the problem in this particular failure (as in this case from_record == table->record[0]), just struck me as something that could easily cause surprises later... Below is a reduced test case that triggers the valgrind error. It did not trigger for me with a debug build, but _did_ trigger (in 5.5) with this build command: EXTRA_FLAGS="-O3 -fno-omit-frame-pointer -Wno-uninitialized" AM_EXTRA_MAKEFLAGS="VERBOSE=1" BUILD/compile-amd64-valgrind-max - Kristian. ----------------------------------------------------------------------- -- source include/have_xtradb.inc set optimizer_switch='mrr=on,mrr_sort_keys=on,index_condition_pushdown=on'; CREATE TABLE t1 ( pk int(11) NOT NULL AUTO_INCREMENT, col_int_key int(11) NOT NULL, col_varchar_key varchar(1) NOT NULL, col_varchar_nokey varchar(1) NOT NULL, PRIMARY KEY (pk), KEY col_int_key (col_int_key), KEY col_varchar_key (col_varchar_key,col_int_key) ) ENGINE=InnoDB; INSERT INTO t1 VALUES (10,8,'v','v'), (11,8,'f','f'), (12,5,'v','v'), (13,8,'s','s'), (14,8,'a','a'), (15,6,'p','p'), (16,7,'z','z'), (17,2,'a','a'), (18,5,'h','h'), (19,7,'h','h'), (20,2,'v','v'), (21,9,'v','v'), (22,142,'b','b'), (23,3,'y','y'), (24,0,'v','v'), (25,3,'m','m'), (26,5,'z','z'), (27,9,'n','n'), (28,1,'d','d'), (29,107,'a','a'); CREATE TABLE t2 ( pk int(11) NOT NULL AUTO_INCREMENT, col_int_key int(11) NOT NULL, col_varchar_key varchar(1) NOT NULL, col_varchar_nokey varchar(1) NOT NULL, PRIMARY KEY (pk), KEY col_int_key (col_int_key), KEY col_varchar_key (col_varchar_key,col_int_key) ) ENGINE=InnoDB; INSERT INTO t2 VALUES (10,3,'i','i'), (11,186,'x','x'), (12,1,'g','g'), (13,8,'q','q'), (14,226,'m','m'), (15,133,'p','p'), (16,6,'e','e'), (17,3,'t','t'), (18,8,'j','j'), (19,5,'h','h'), (20,7,'w','w'); set join_cache_level=6; set join_buffer_size=1536; SELECT count(*), sum(table1.col_int_key*table2.pk) FROM t2 AS table1, t1 AS table2, t2 AS table3 WHERE table3.col_varchar_nokey = table2.col_varchar_key AND table3.pk > table2.col_varchar_nokey ; -----------------------------------------------------------------------

1 0

Re: [Maria-developers] Rev 4323: Set proper alternative names on RHEL7: they should match Fedora names
by Sergei Golubchik 30 Oct '14

30 Oct '14

Hi, Sergey! I cannot verify whether rhel7 alternative names should match Fedora names or they should be something different. Assuming they should match, your patch is ok. On Oct 09, Sergey Vojtovich wrote: > ------------------------------------------------------------ > revno: 4323 > revision-id: svoj(a)mariadb.org-20141009112748-eq3xwoyobgnz0cxj > parent: svoj(a)mariadb.org-20141008141031-xv2x7vrq269202vf > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 5.5 > timestamp: Thu 2014-10-09 15:27:48 +0400 > message: > Set proper alternative names on RHEL7: they should match Fedora names. > === modified file 'cmake/cpack_rpm.cmake' > --- a/cmake/cpack_rpm.cmake 2014-10-08 07:35:00 +0000 > +++ b/cmake/cpack_rpm.cmake 2014-10-09 11:27:48 +0000 > @@ -182,7 +182,7 @@ IF(RPM MATCHES "(rhel|centos)5") > ELSEIF(RPM MATCHES "(rhel|centos)6") > ALTERNATIVE_NAME("client" "mysql") > ALTERNATIVE_NAME("shared" "mysql-libs") > -ELSEIF(RPM MATCHES "fedora") > +ELSEIF(RPM MATCHES "fedora" OR RPM MATCHES "(rhel|centos)7") > SET(epoch 1) # this is fedora > ALTERNATIVE_NAME("client" "mariadb") > ALTERNATIVE_NAME("client" "mysql") Regards, Sergei

2 2

[Maria-developers] Fwd: [debian-mysql] Backporting the mysql_no_login plugin
by Colin Charles 30 Oct '14

30 Oct '14

This seems to get a +1 for backporting by at least Honza (RH), so I am wondering if we do this in the 5.5 branch too, that is shipping in many distributions. Begin forwarded message: > From: "Norvald H. Ryeng" <norvald.ryeng(a)oracle.com> > Subject: [debian-mysql] Backporting the mysql_no_login plugin > Date: 24 October 2014 15:49:34 GMT+8 > To: "pkg-mysql-maint(a)lists.alioth.debian.org" <pkg-mysql-maint(a)lists.alioth.debian.org>, "Honza Horak" <hhorak(a)redhat.com>, "Roman Drahtmueller" <draht(a)suse.de> > > Hi package maintainers, > > We have a new plugin in MySQL 5.7 that makes it possible to have > accounts that can't log in: > > CREATE USER foo@localhost IDENTIFIED WITH 'mysql_no_login'; > > The mysql_no_login plugin simply denies all login attempts. This is > useful for users that are created, e.g., to serve as proxy users, or > as owners of stored programs/functions, views or events. > > This new plugin doesn't fix known security defects in the server, but > does provide new and better means to harden security. Best practices > for security include application of least-required privileges, and in > some cases, that means no client connections for privileged > accounts. This new plugin provides means to implement such > restrictions in a standard way. > > Because of the security benefits, we'd like to discuss backporting it > to 5.6. Like you, we don't like big changes to GA releases, but this > time we think it has a good use case, it's safe and has a very low > risk of regressions: > > - Since this is a plugin, it doesn't touch server code > - All new code is in a plugin that must be enabled explicitly by the > DBA > - The code itself is very simple. It's only one line of "real" code > (unconditionally return authentication failure), plus necessary > plugin plumbing to fill out the plugin API. > > If we backport this to 5.6, there are multiple ways to avoid it: > > - Apply a patch from us to remove the plugin > - Don't build it > - Build it, but don't ship it > - Build and ship it, but don't use it (in any case, the DBA has to > enable it and alter the user accounts to use it) > > So what do you think about backporting this? The only thing you'll > notice is one more file in the plugins directory. > > Regards, > > Norvald > > _______________________________________________ > pkg-mysql-maint mailing list > pkg-mysql-maint(a)lists.alioth.debian.org > http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-mysql-maint -- Colin Charles, Chief Evangelist, MariaDB Corporation blog: http://bytebot.net/blog/| t: +6-012-204-3201 | Skype: colincharles

2 1

Re: [Maria-developers] [Commits] b28e6bf: MDEV-6945 MDEV-6948 MDEV-6956 MDEV-6957 MDEV-6953 MDEV-6954 MDEV-6955
by Sergei Golubchik 29 Oct '14

29 Oct '14

Hi, Sanja! On Oct 29, sanja(a)mariadb.com wrote: > --- > mysql-test/r/set_statement.result | 14 ++++++++++++++ > mysql-test/r/set_statement_profiling.result | 4 ++++ > mysql-test/t/set_statement.test | 18 ++++++++++++++++++ > mysql-test/t/set_statement_profiling.test | 9 +++++++++ Did you put set_statement_debug.test in a separate commit? That's good. > sql/set_var.h | 4 +++- > sql/share/errmsg-utf8.txt | 2 ++ > sql/sql_parse.cc | 5 +++++ > sql/sys_vars.cc | 20 +++++++++++--------- > sql/sys_vars.h | 1 + > 9 files changed, 67 insertions(+), 10 deletions(-) > > diff --git a/sql/share/errmsg-utf8.txt b/sql/share/errmsg-utf8.txt > index bb42807..1633169 100644 > --- a/sql/share/errmsg-utf8.txt > +++ b/sql/share/errmsg-utf8.txt > @@ -7113,3 +7113,5 @@ ER_STATEMENT_TIMEOUT 70100 > eng "Query execution was interrupted (max_statement_time exceeded)" > ER_SUBQUERIES_NOT_SUPPORTED 42000 > eng "%s does not support subqueries or stored functions." > +ER_SET_STATEMENT_NOT_SUPPORTED 42000 > + eng "Variable '%s' is not supported in SET STATEMENT." Use the text eng "The system variable %.200s cannot be set in SET STATEMENT." > diff --git a/sql/sys_vars.cc b/sql/sys_vars.cc > index 6252e89..fd77e5f 100644 > --- a/sql/sys_vars.cc > +++ b/sql/sys_vars.cc > @@ -997,7 +997,7 @@ static bool check_master_connection(sys_var *self, THD *thd, set_var *var) > static Sys_var_session_lexstring Sys_default_master_connection( > "default_master_connection", > "Master connection to use for all slave variables and slave commands", > - SESSION_ONLY(default_master_connection), > + NO_SET_STMT SESSION_ONLY(default_master_connection), Why should this be prohibited? I'd rather have it supported. Use case: setting many connection variables for a single master, like in SET STATEMENT default_master_connection=foobar SET max_relay_log_size=10000000, sql_slave_skip_counter=5; Admittedly, this use case, while valid, isn't a strong argument at the moment - we don't have many per-master variables yet. When we'll have more, this will become more important. So, if fixing default_master_connection is not easy - keep it NO_SET_STMT, it can be enabled later. Why, exactly, does it fail? > NO_CMD_LINE, IN_SYSTEM_CHARSET, > DEFAULT(""), MAX_CONNECTION_NAME, ON_CHECK(check_master_connection)); > #endif > @@ -3527,12 +3528,12 @@ static Sys_var_bit Sys_unique_checks( > #ifdef ENABLED_PROFILING > static Sys_var_bit Sys_profiling( > "profiling", "profiling", > - SESSION_VAR(option_bits), NO_CMD_LINE, OPTION_PROFILING, > + NO_SET_STMT SESSION_VAR(option_bits), NO_CMD_LINE, OPTION_PROFILING, Why should this be prohibited? I'd rather have it supported. Use case: one usually needs to profile a single statement, not everything. > DEFAULT(FALSE)); > > static Sys_var_ulong Sys_profiling_history_size( > "profiling_history_size", "Limit of query profiling memory", > - SESSION_VAR(profiling_history_size), CMD_LINE(REQUIRED_ARG), > + NO_SET_STMT SESSION_VAR(profiling_history_size), CMD_LINE(REQUIRED_ARG), > VALID_RANGE(0, 100), DEFAULT(15), BLOCK_SIZE(1)); > #endif > > @@ -3566,7 +3567,8 @@ static bool check_skip_replication(sys_var *self, THD *thd, set_var *var) > > static Sys_var_bit Sys_skip_replication( > "skip_replication", "skip_replication", > - SESSION_ONLY(option_bits), NO_CMD_LINE, OPTION_SKIP_REPLICATION, > + NO_SET_STMT SESSION_ONLY(option_bits), > + NO_CMD_LINE, OPTION_SKIP_REPLICATION, Why should this be prohibited? I'd rather have it supported. Use case: skip replication of a single statement. A generic version of FLUSH NO_WRITE_TO_BINLOG TABLE REPAIR NO_WRITE_TO_BINLOG TABLE ANALYZE NO_WRITE_TO_BINLOG TABLE OPTIMIZE NO_WRITE_TO_BINLOG TABLE and all ALTER TABLE ... PARTITION variants: ALTER TABLE ... REBUILD PARTITION NO_WRITE_TO_BINLOG ALTER TABLE ... OPTIMIZE PARTITION NO_WRITE_TO_BINLOG ALTER TABLE ... ANALYZE PARTITION NO_WRITE_TO_BINLOG ALTER TABLE ... REPAIR PARTITION NO_WRITE_TO_BINLOG ALTER TABLE ... COALESCE PARTITION NO_WRITE_TO_BINLOG ALTER TABLE ... ADD PARTITION NO_WRITE_TO_BINLOG ALTER TABLE ... REORGANIZE PARTITION NO_WRITE_TO_BINLOG > DEFAULT(FALSE), NO_MUTEX_GUARD, NOT_IN_BINLOG, > ON_CHECK(check_skip_replication)); > > @@ -3688,7 +3690,7 @@ static ulonglong read_rand_seed(THD *thd) > static Sys_var_session_special Sys_rand_seed1( > "rand_seed1", "Sets the internal state of the RAND() " > "generator for replication purposes", > - sys_var::ONLY_SESSION, NO_CMD_LINE, > + NO_SET_STMT sys_var::ONLY_SESSION, NO_CMD_LINE, > VALID_RANGE(0, ULONG_MAX), BLOCK_SIZE(1), > NO_MUTEX_GUARD, IN_BINLOG, ON_CHECK(0), > ON_UPDATE(update_rand_seed1), ON_READ(read_rand_seed)); > @@ -3706,7 +3708,7 @@ static bool update_rand_seed2(THD *thd, set_var *var) > static Sys_var_session_special Sys_rand_seed2( > "rand_seed2", "Sets the internal state of the RAND() " > "generator for replication purposes", > - sys_var::ONLY_SESSION, NO_CMD_LINE, > + NO_SET_STMT sys_var::ONLY_SESSION, NO_CMD_LINE, > VALID_RANGE(0, ULONG_MAX), BLOCK_SIZE(1), > NO_MUTEX_GUARD, IN_BINLOG, ON_CHECK(0), > ON_UPDATE(update_rand_seed2), ON_READ(read_rand_seed)); I'd rather fix rand_seed1/rand_seed2 variables not to be special anymore. Like, move them to SV structure, remove special behavior and so on. Regards, Sergei

1 0

Re: [Maria-developers] [Commits] b18ddd2: Added 'mariadb-galera-server' metapackage to install all
by Sergei Golubchik 28 Oct '14

28 Oct '14

Hi, Nirbhay! On Oct 23, nirbhay(a)skysql.com wrote: > > Added 'mariadb-galera-server' metapackage to install all > necessary Galera-related dependencies. > > --- > debian/dist/Debian/control | 13 ++++++++++--- > debian/dist/Ubuntu/control | 13 ++++++++++--- > 2 files changed, 20 insertions(+), 6 deletions(-) > > diff --git a/debian/dist/Debian/control b/debian/dist/Debian/control > index e68f027..bc48dbc 100644 > --- a/debian/dist/Debian/control > +++ b/debian/dist/Debian/control > @@ -194,9 +194,7 @@ Recommends: libhtml-template-perl > Pre-Depends: mariadb-common, adduser (>= 3.40), debconf > Depends: mariadb-client-10.1 (>= ${source:Version}), libdbi-perl, > perl (>= 5.6), ${shlibs:Depends}, ${misc:Depends}, psmisc, passwd, > - lsb-base (>= 3.0-10), mariadb-server-core-10.1 (>= ${binary:Version}), > - galera (>=25.2), rsync, lsof, socat | netcat, grep, gawk, iproute, > - coreutils, findutils > + lsb-base (>= 3.0-10), mariadb-server-core-10.1 (>= ${binary:Version}) > Provides: mariadb-server, mysql-server, virtual-mysql-server > Conflicts: mariadb-server (<< ${source:Version}), mysql-server (<< ${source:Version}), > mysql-server-4.1, mysql-server-5.0, mysql-server-5.1, mysql-server-5.5, Looks ok. Few comments: 1. What about RPM? 2. Did you compare with, say, non-galera 10.0 - just to make sure all dependencies you removed are, indeed, only for galera and the server doesn't need any of that? 3. I'd also add /etc/mysql/conf.d/galera.cnf (/etc/my.cnf.d/galera.cnf in rpms) that enables wsrep, and may be sets other reasonable defaults for galera. Regards, Sergei

1 0

[Maria-developers] DisconnectEx() problem
by Delveri chick 28 Oct '14

28 Oct '14

Hi i was tracing why mysql_socket_shutdown(...) won't call DisconnectEx(...) as reported in the TODO list and realised DisconnectEx(...) cannot be called in include/mysql/psi/mysql_socket.h. Possible solution would be to add it here or include the *.h where it it defined. Where is it defined please? Regards Delveri

2 1

Re: [Maria-developers] [Commits] 18a7470: MDEV-6939 : Dots in file names of configuration files
by Sergei Golubchik 28 Oct '14

28 Oct '14

Hi, Nirbhay! 1. Why 10.1? 1.0 would be fine too, may be even 5.5 2. In cases like this I usually make two commits, one for a cleanup and one for a bug fix. Anyway, the fix is ok to push, of course. On Oct 24, nirbhay(a)mariadb.com wrote: > revision-id: 18a7470a563c30bffbfe787e4e5406b687597a44 > parent(s): cc656e4d8538326d85e84adbe48d0a3739d55674 > committer: Nirbhay Choubey > branch nick: server > timestamp: 2014-10-24 16:34:54 -0400 > message: > > MDEV-6939 : Dots in file names of configuration files > > Use fn_ext2() to get the file extension from last occurrence > of FN_EXTCHAR ('.') instead. Also made some cosmetic changes > in mysys/mf_fn_ext.c. > > --- > mysys/mf_fn_ext.c | 11 ++++++----- > mysys/my_default.c | 2 +- > 2 files changed, 7 insertions(+), 6 deletions(-) > Regards, Sergei

1 0

[Maria-developers] DATE and GROUP_MIN_MAX optimization
by Alexander Barkov 28 Oct '14

28 Oct '14

Hello Sergey, I'm working on pluggable data types and moving some pieces of the code into new classes for data type handlers. I noticed that GROUP_MIN_MAX optimization works fine with a TIME column with temporal and string literals but does not work for an integer literal: # This script creates demo data: DROP TABLE IF EXISTS t1; CREATE TABLE t1 (id INT, b TIME, KEY(id,b)) ENGINE=MyISAM; INSERT INTO t1 VALUES (1,10),(1,11),(1,12),(1,13),(1,14),(1,15),(1,16),(2,10),(2,11),(2,12),(2,13),(2,14),(2,15),(2,16); SELECT * FROM t1; # Now three EXPLAIN queries: explain SELECT id, MIN(b),MAX(b) FROM t1 WHERE b<TIME'10:10:10' GROUP BY id; explain SELECT id, MIN(b),MAX(b) FROM t1 WHERE b<'10:10:10' GROUP BY id; explain SELECT id, MIN(b),MAX(b) FROM t1 WHERE b<10 GROUP BY id; The first and the second EXPLAIN queries tell that 8 rows are to be examined; tracing the code in debugger shows that group_min_max optimization is active: > mysql> explain SELECT id, MIN(b),MAX(b) FROM t1 WHERE b<TIME'10:10:10' GROUP BY id; > +------+-------------+-------+-------+---------------+------+---------+------+------+---------------------------------------+ > | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | > +------+-------------+-------+-------+---------------+------+---------+------+------+---------------------------------------+ > | 1 | SIMPLE | t1 | range | NULL | id | 9 | NULL | 8 | Using where; Using index for group-by | > +------+-------------+-------+-------+---------------+------+---------+------+------+---------------------------------------+ > 1 row in set (0.00 sec) > > mysql> explain SELECT id, MIN(b),MAX(b) FROM t1 WHERE b<'10:10:10' GROUP BY id; > +------+-------------+-------+-------+---------------+------+---------+------+------+---------------------------------------+ > | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | > +------+-------------+-------+-------+---------------+------+---------+------+------+---------------------------------------+ > | 1 | SIMPLE | t1 | range | NULL | id | 9 | NULL | 8 | Using where; Using index for group-by | > +------+-------------+-------+-------+---------------+------+---------+------+------+---------------------------------------+ > 1 row in set (0.00 sec) > The third EXPLAIN query tells that all 14 rows is to be examined, and tracing in debugging shows that group_min_max is not enabled: > mysql> explain SELECT id, MIN(b),MAX(b) FROM t1 WHERE b<10 GROUP BY id; > +------+-------------+-------+-------+---------------+------+---------+------+------+--------------------------+ > | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | > +------+-------------+-------+-------+---------------+------+---------+------+------+--------------------------+ > | 1 | SIMPLE | t1 | index | NULL | id | 9 | NULL | 14 | Using where; Using index | > +------+-------------+-------+-------+---------------+------+---------+------+------+--------------------------+ > 1 row in set (0.01 sec) Any reasons not to use group_min_max for TIME column in a combination with an integer literal? I have a feeling that it should work. What about decimal and real literals? Thanks.

1 0

[Maria-developers] Problem with KVM version in Buildbot?
by Kristian Nielsen 24 Oct '14

24 Oct '14

Hi Daniel, We have a problem with a build in Buildbot: https://internal.askmonty.org/buildbot/builders/kvm-rpm-centos7_0-x86_64/bu… + qemu-img create -b /kvm/vms/vm-centos7_0-x86_64-build.qcow2 -f qcow2 /dev/shm/vm-tmp-2302.qcow2 qemu-img: '' uses a qcow2 feature which is not supported by this qemu version: QCOW version 3 qemu-img: Could not open '/kvm/vms/vm-centos7_0-x86_64-build.qcow2' Failed to clone base image, aborting This failure happens on the build host `terrier'. I assume the problem is that the image was created on a machine with a newer KVM? The host terrier seems to have a rather old one: knielsen@terrier:~> kvm --version QEMU emulator version 0.13.0 (qemu-kvm-devel), Copyright (c) 2003-2008 Fabrice Bellard How do you think we should solve the problem? Should we upgrade terrier? Or should we alternatively make sure to create new images in a backward-compatible version (and re-create affected images) ? Also, another issue related to VM images. Our old images were created with rather small disk size. I noticed that some new images are created much larger, for example: $ qemu-img info /kvm/vms/vm-sid-amd64-build.qcow2 image: /kvm/vms/vm-sid-amd64-build.qcow2 file format: qcow2 virtual size: 40G (42949672960 bytes) disk size: 3.1G cluster_size: 65536 Format specific information: compat: 0.10 That is fine, I think, it is indeed annoying when we run out of space in a build VM. However, the reason I noticed this is because host bb02 ran out of space in /run/shm/ due to a 16 GB copy of this image. So when we use larger VM images, we cannot use the optimisation to have the temporary copies in ramdisk, lest we get disk full errors. If the virtual size is larger to (ram disk size) / (max_builds), then we need to disable using ramdisk for buildbot for that image. I will try to fix the config to not use ramdisk for existing images that have too large virtual size. But it is something to keep in mind for the future. - Kristian.

3 3

Re: [Maria-developers] SQL Accelerator FPGA/ASIC
by Nasser Ghoseiri 24 Oct '14

24 Oct '14

Dear Jan, Thank you for your response. I have background in SQL (MS, Sybase, MySQL, Postgre), so luckily I'm not a beginner in query writing, however, I have absolutely no background in how databases work on background (other than hash-table). The idea is not really SIMD here, but a specially designed hardware to perform database functions much faster than the conventional DISK<->RAM<->CACHE<->CPU model. What I'm struggling to figure is that the latency caused by a enterprise-grade SQL script (not very correct and advanced code, but not very low-quality either), is Memory-Related or Arithmetic related or both? Here is what can be made for example: 1) A card with 16 DDR3 Chips on board and a Central FPGA, allowing very fast access to data, and like 100 small CPUs integrated in the FPGA to process a query. (a query that was written to take advantage of these features however) 2) A card with XDR Memory chips and like 32GB of Flash Chips to achieve very high-speed storage, retrieval (SSD<->RAM) and an FPGA that has many cores inside. A query is executed by FPGA pulling large chunk of data from Flashes, putting them in XDR RAM chips and processes them in a parallel manner 3) A card with XDR Memory chips and an FPGA on board, connected to a PCI Express 8x which receives all the data it needs from the host computer, and does the parallel processing inside itself, returning it back to the host. 4) A card with an FPGA inside, having 100 small CPUs and PCI Express to take commands from host computer and process in a parallel manner 5) A card with an FPGA, not having small processors inside, but special circuitry that does either lookup, sorting, etc in a very fast manner These are a few examples that I gave. The difference among them is the notion of having RAM, having a cold-storage, having an FPGA with many CPUs inside or an FPGA with special dedicated circuitry that performs special functions in parallel that is difficult for a CPU to handle (imaging adding 300 numbers to another 300 numbers in a single cycle, where a CPU with SIMD would take much longer, or maybe a string processor that processes many string columns with PATINDEX, etc in parallel where a CPU would be slow as it can't handle them in parallel). So what I'm researching is whether beneath the SQL pyramid, there is a memory intensive operation going on or logic-intensive. What to look for and what to aim for... Thanks in advance for your help, Nasser On Friday, October 24, 2014 7:49 PM, Jan Lindström <jan.lindstrom(a)mariadb.com> wrote: Hi, This idea of SIMD (single instruction multiple data) processing is not totally new one, similarly the idea to perform SQL-operations inside GPU or GPGA is not new. In traditional relational databases problematic is the fact that e.g. in your example TableX contains several columns, picking columns A and B from pages that reside first on disk, then on main-memory and finally on L1-L3 cache is not cheap, and then they are not on continuous memory. This is because page in cache would contain values for other columns that we not even need. In columnar database architecture this would be a lot easier, you just feed column containing values for A and B directly to SIMD operation and every page in main memory would contain a lot more values to process compared to traditional relational database where page would contain also values for columns that we really do not even need to process the result set of query. Anyway, I find the proposal interesting and challenging. R: Jan Lindström On Thu, Oct 23, 2014 at 3:39 PM, Nasser Ghoseiri <cdmcsd(a)yahoo.com> wrote: Dear Serg, > > >Following our little chat in IRC, I'm writing this email to explain in more detail what the idea is. My name is Nasser GHOSEIR (Founder, CTO of Butterfly labs), which I must note that this project is not from Butterfly Labs, >but will be a new company in Europe. > > >Our idea is to find a way to accelerate SQL query processing by either: > > >1) Creating an FPGA solution (Which will later evolve into ASIC), that has like 400 processors in itself, allow distribute calculation of some kind >2) Creating an FPGA/ASIC solution that performs large number of unrelated tasks (such as addition, multiplication, etc) in parallel >3) Creating an FPGA/ASIC solution that allows very high-speed access to data with some PRE-PROCESSING involved to accelerate the calculation >4) Creating a very high-speed storage solution, but no pre-processing. > > >An example to give is for 200,000 rows, imagine: "SELECT A+B, C FROM TableX WHERE C > 0". If there are 200,000 records, the processor has to perform 200,000 additions (A+B). >A CPU will handle these additions one-by-one (or maybe few-by-few if SSE, etc is used). However, an FPGA/ASIC solution can perform 1000 additions in a single-cycle. This results in acceleration. > > >Now, to what extend this can be effective, or what other solutions (maybe string processing?) can be implemented to accelerate the SQL processing is a question to me right now. But we do have experience in >making extremely fast processors (our Butterfly Labs Monarch chip performs 400 billion double-SHA256 hashes in 1 second, around 20,000 times faster than the best intel XEON processor can do only twenty >million hashes per second). > > >The project is in brain-storming phase, and we are aware that this solution will be useful to large enterprises, or companies that have to deal with 200,000 rows in a single "SELECT" query. The idea is to integrate >some features in hardware, and then re-write a portion of MariaDB to take advantage of the new resources. Also, it is possible that users/companies will need to re-write their query to make it compliant. > > >You can reach my by mail at cdmcsd(a)yahoo.com or by phone at +33 6 72 17 26 19 (France). > > > > > > >Best Regards, >Nasser GHOSEIRI > > > > > > > > >_______________________________________________ >Mailing list: https://launchpad.net/~maria-developers >Post to : maria-developers(a)lists.launchpad.net >Unsubscribe : https://launchpad.net/~maria-developers >More help : https://help.launchpad.net/ListHelp > >

1 0

[Maria-developers] Please review: MDEV-5528 Command line variable to choose MariaDB-5.3 vs MySQL-5.6 temporal data formats
by Alexander Barkov 24 Oct '14

24 Oct '14

Hi, Please review my patch for MDEV-5528. It also fixes a bug: MDEV-6649 Different warnings for TIME and TIME(N) when @@old_mode=zero_date_time_cast Thanks.

2 8

[Maria-developers] doubt about mariadb dialog plugin at server and client side
by Roberto Spadim 23 Oct '14

23 Oct '14

Hi guys I'm with a doubt at dialog plugin I created a user at mariadb server: ---SQL INSTALL PLUGIN three_attempts SONAME 'dialog_examples.so'; CREATE USER test_dialog IDENTIFIED VIA three_attempts USING 'SECRET'; FLUSH PRIVILEGES; --- now i'm trying to connect as user test_dialog: ---SHELL $mysql *--plugin-dir=/home/rspadim/mariadb/mariadb-10.0.14/plugin/auth_dialog* --user=test_dialog --host=spd1.spadim.com.br --password Enter password: (i just hit enter) [mariadb] Password, please: (enter again) [mariadb] Password, please: (SECRET enter) Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 49243 Server version: 5.5.5-10.0.14-MariaDB-log Source distribution Copyright (c) 2000, 2012, Oracle, Monty Program Ab and others. Reading history-file /home/beto/.mysql_history Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> quit Writing history-file /home/beto/.mysql_history --- ---SHELL $ls /home/rspadim/mariadb/mariadb-10.0.14/plugin/auth_dialog CMakeFiles cmake_install.cmake CMakeLists.txt CTestTestfile.cmake dialog.c *dialog.so* Makefile --- ---MYSQL.USER table || *Host* || *User* || *Password* || *Select_priv* || *Insert_priv* || *Update_priv* || *Delete_priv* || *Create_priv* || *Drop_priv* || *Reload_priv* || *Shutdown_priv* || *Process_priv* || *File_priv* || *Grant_priv* || *References_priv* || *Index_priv* || *Alter_priv* || *Show_db_priv* || *Super_priv* || *Create_tmp_table_priv* || *Lock_tables_priv* || *Execute_priv* || *Repl_slave_priv* || *Repl_client_priv* || *Create_view_priv* || *Show_view_priv* || *Create_routine_priv* || *Alter_routine_priv* || *Create_user_priv* || *Event_priv* || *Trigger_priv* || *Create_tablespace_priv* || *ssl_type* || *ssl_cipher* || *x509_issuer* || *x509_subject* || *max_questions* || *max_updates* || *max_connections* || *max_user_connections* || *plugin* || *auth_string* || || % || test_dialog || >>>CHECK NO PASSWORD!<<< || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || || || || || 0 || 0 || 0 || 0 || three_attempts || SECRET || --- the doubts: 1) why it accept my login with password? i *don't* have a password at mysql.user schema 2) why i didn't received a dialog like in heidisql? 3) there's a auth trace or auth verbose option at client side? or any other command to check if i loaded the dialog plugin? -- Roberto Spadim

2 3

[Maria-developers] SQL Accelerator FPGA/ASIC
by Nasser Ghoseiri 23 Oct '14

23 Oct '14

Dear Serg, Following our little chat in IRC, I'm writing this email to explain in more detail what the idea is. My name is Nasser GHOSEIR (Founder, CTO of Butterfly labs), which I must note that this project is not from Butterfly Labs, but will be a new company in Europe. Our idea is to find a way to accelerate SQL query processing by either: 1) Creating an FPGA solution (Which will later evolve into ASIC), that has like 400 processors in itself, allow distribute calculation of some kind 2) Creating an FPGA/ASIC solution that performs large number of unrelated tasks (such as addition, multiplication, etc) in parallel 3) Creating an FPGA/ASIC solution that allows very high-speed access to data with some PRE-PROCESSING involved to accelerate the calculation 4) Creating a very high-speed storage solution, but no pre-processing. An example to give is for 200,000 rows, imagine: "SELECT A+B, C FROM TableX WHERE C > 0". If there are 200,000 records, the processor has to perform 200,000 additions (A+B). A CPU will handle these additions one-by-one (or maybe few-by-few if SSE, etc is used). However, an FPGA/ASIC solution can perform 1000 additions in a single-cycle. This results in acceleration. Now, to what extend this can be effective, or what other solutions (maybe string processing?) can be implemented to accelerate the SQL processing is a question to me right now. But we do have experience in making extremely fast processors (our Butterfly Labs Monarch chip performs 400 billion double-SHA256 hashes in 1 second, around 20,000 times faster than the best intel XEON processor can do only twenty million hashes per second). The project is in brain-storming phase, and we are aware that this solution will be useful to large enterprises, or companies that have to deal with 200,000 rows in a single "SELECT" query. The idea is to integrate some features in hardware, and then re-write a portion of MariaDB to take advantage of the new resources. Also, it is possible that users/companies will need to re-write their query to make it compliant. You can reach my by mail at cdmcsd(a)yahoo.com or by phone at +33 6 72 17 26 19 (France). Best Regards, Nasser GHOSEIRI

1 0

[Maria-developers] COMPOUND STATEMENT fails on Windows.
by Peter Laursen 22 Oct '14

22 Oct '14

I tried to comment on this Blog https://blog.mariadb.org/mariadb-10-1-1-compound-statements/, but it won't let me use my (corporate) gmail address for authencticaton. So I send this mail instead: I tried the example provided on Windows (both server and client are Win7 64 bit) using the command line client shipped with MariaDB 10.1: Enter password: ******** Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 5 Server version: 10.1.1-MariaDB mariadb.org binary distribution Copyright (c) 2000, 2014, Oracle, SkySQL Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> use test; Database changed MariaDB [test]> IF @have_csv = 'YES' THEN -> CREATE TABLE IF NOT EXISTS general_log ( -> event_time TIMESTAMP(6) NOT NULL, -> user_host MEDIUMTEXT NOT NULL, -> thread_id BIGINT(21) UNSIGNED NOT NULL, -> server_id INTEGER UNSIGNED NOT NULL, -> command_type VARCHAR(64) NOT NULL, -> argument MEDIUMTEXT NOT NULL -> ) engine=CSV CHARACTER SET utf8 comment="General log"; ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' a t line 9 MariaDB [test]> END IF; Obviously the SEMICOLON after comment="General log" disturbs. But this is no better: MariaDB [test]> IF @have_csv = 'YES' THEN -> CREATE TABLE IF NOT EXISTS general_log ( -> event_time TIMESTAMP(6) NOT NULL, -> user_host MEDIUMTEXT NOT NULL, -> thread_id BIGINT(21) UNSIGNED NOT NULL, -> server_id INTEGER UNSIGNED NOT NULL, -> command_type VARCHAR(64) NOT NULL, -> argument MEDIUMTEXT NOT NULL -> ) engine=CSV CHARACTER SET utf8 comment="General log" -> END IF; ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'END IF' at line 10 MariaDB [test]> What is the problem here? Is it someWindows-specific bug/issue? BTW it is the same in SQLyog (compiled with MariaDB C-API) as in commandline. -- Peter -- Webyog

2 2

Re: [Maria-developers] [Commits] 28fcccc: MDEV-5231: Per query variables from Percona Server (rewritten)
by Sergei Golubchik 22 Oct '14

22 Oct '14

Hi, Sanja! On Oct 13, sanja(a)mariadb.com wrote: > revision-id: 28fccccad32fa8a47dc572b968613e18ff51b634 > parent(s): fec5ab5a56cb9a45c621207620cc85079cddf537 > committer: Oleksandr Byelkin > branch nick: work-maria-10.1-sysvar2 > timestamp: 2014-10-13 22:35:40 +0200 > message: > > MDEV-5231: Per query variables from Percona Server (rewritten) This looks a lot better that the previous patch. Thanks! See my comments below. Some parts of the patch are not used (left-overs from the old implementation?). Some others, I think, are not needed. And some of the old tests don't make much sense, your new tests made them redundant and useless. A couple of comments after reading my previous review: * please add a test for any sysvar with non-trivial implementations of sys_var::session_update(), that is, for a sysvar where old approach doesn't work. Unless, of course, your timestamp tests are exactly that :) * add tests with @@default_engine. For example: SET @@default_engine=MyISAM; SET STATEMENT @@default_engine=MEMORY CREATE TABLE t1 (a int); SHOW CREATE TABLE t1; -- verify the engine SET STATEMENT @@default_engine=MyISAM CREATE TABLE t2 (a int); etc. See other comments below: > diff --git a/mysql-test/r/set_statement.result b/mysql-test/r/set_statement.result > new file mode 100644 > index 0000000..fed937e > --- /dev/null > +++ b/mysql-test/r/set_statement.result > @@ -0,0 +1,941 @@ > +'# SET STATEMENT ..... FOR .... TEST' > +DROP TABLE IF EXISTS t1; > +DROP FUNCTION IF EXISTS myProc; > +DROP PROCEDURE IF EXISTS p1; > +DROP PROCEDURE IF EXISTS p2; > +DROP PROCEDURE IF EXISTS p3; > +DROP PROCEDURE IF EXISTS p4; > +DROP PROCEDURE IF EXISTS p5; > +DROP TABLE IF EXISTS STATEMENT; > +'# Setup database' > +CREATE TABLE t1 (v1 INT, v2 INT); > +INSERT INTO t1 VALUES (1,2); > +INSERT INTO t1 VALUES (3,4); > +'' > +'#------------------ STATEMENT Test 1 -----------------------#' > +'# Initialize variables to known setting' > +SET SESSION sort_buffer_size=100000; > +'' > +'# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +SET STATEMENT sort_buffer_size=150000 FOR SELECT *,@@sort_buffer_size FROM t1; > +v1 v2 @@sort_buffer_size > +1 2 150000 > +3 4 150000 > +'' > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +'' > +'#------------------ STATEMENT Test 2 -----------------------#' > +'# Initialize variables to known setting' > +SET SESSION binlog_format=mixed; > +SET SESSION sort_buffer_size=100000; > +'# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +SHOW SESSION VARIABLES LIKE 'binlog_format'; > +Variable_name Value > +binlog_format MIXED > +SET STATEMENT sort_buffer_size=150000, binlog_format=row > +FOR SELECT *,@@sort_buffer_size,@@binlog_format FROM t1; > +v1 v2 @@sort_buffer_size @@binlog_format > +1 2 150000 ROW > +3 4 150000 ROW > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +SHOW SESSION VARIABLES LIKE 'binlog_format'; > +Variable_name Value > +binlog_format MIXED > +'' > +'#------------------ STATEMENT Test 3 -----------------------#' > +'# set initial variable value, make prepared statement > +SET SESSION binlog_format=row; > +PREPARE stmt1 FROM 'SET STATEMENT binlog_format=row FOR SELECT *,@@binlog_format FROM t1'; > +'' > +'# Change variable setting' > +SET SESSION binlog_format=mixed; > +'' > +'# Pre-STATEMENT variable value' > +'' > +SHOW SESSION VARIABLES LIKE 'binlog_format'; > +Variable_name Value > +binlog_format MIXED > +'' > +EXECUTE stmt1; > +v1 v2 @@binlog_format > +1 2 ROW > +3 4 ROW > +'' > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'binlog_format'; > +Variable_name Value > +binlog_format MIXED > +'' > +DEALLOCATE PREPARE stmt1; > +'#------------------ STATEMENT Test 4 -----------------------#' > +'# set initial variable value, make prepared statement > +SET SESSION myisam_sort_buffer_size=500000, myisam_repair_threads=1; > +'' > +'# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'myisam_sort_buffer_size'; > +Variable_name Value > +myisam_sort_buffer_size 500000 > +SHOW SESSION VARIABLES LIKE 'myisam_repair_threads'; > +Variable_name Value > +myisam_repair_threads 1 > +'' > +SET STATEMENT myisam_sort_buffer_size=800000, > +myisam_repair_threads=2 FOR OPTIMIZE TABLE t1; 1. this is not "make prepared statement" as the comment says 2. what does it actually test? you don't verify whether new values had any effect. > +Table Op Msg_type Msg_text > +test.t1 optimize status OK > +'' > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'myisam_sort_buffer_size'; > +Variable_name Value > +myisam_sort_buffer_size 500000 > +SHOW SESSION VARIABLES LIKE 'myisam_repair_threads'; > +Variable_name Value > +myisam_repair_threads 1 > +'' > +'#------------------ STATEMENT Test 5 -----------------------#' > +'# Initialize variables to known setting' > +SET SESSION sort_buffer_size=100000; > +'' > +'# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +'' > +SET STATEMENT sort_buffer_size=150000 FOR SELECT * FROM t2; > +ERROR 42S02: Table 'test.t2' doesn't exist > +'' > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 150000 Looks wrong. The value is not restored in case of an error. > +'' > +'#------------------ STATEMENT Test 6 -----------------------#' > +'# Initialize variables to known setting' > +SET SESSION keep_files_on_create=ON; > +'' > +'# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'keep_files_on_create'; > +Variable_name Value > +keep_files_on_create ON > +'' > +SET STATEMENT keep_files_on_create=OFF FOR SELECT *,@@keep_files_on_create FROM t1; > +v1 v2 @@keep_files_on_create > +1 2 0 > +3 4 0 How is this test different from other tests above? > +'' > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'keep_files_on_create'; > +Variable_name Value > +keep_files_on_create ON > +'' > +'#------------------ STATEMENT Test 7 -----------------------#' > +'# Initialize variables to known setting' > +SET SESSION max_join_size=2222220000000; > +'' > +'# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'max_join_size'; > +Variable_name Value > +max_join_size 2222220000000 > +'' > +SET STATEMENT max_join_size=1000000000000 FOR SELECT *,@@max_join_size FROM t1; > +v1 v2 @@max_join_size > +1 2 1000000000000 > +3 4 1000000000000 Same question - how is this test different from other tests above? > +'' > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'max_join_size'; > +Variable_name Value > +max_join_size 2222220000000 > +'' > +'#------------------Test 8-----------------------#' > +'# Initialize test variables' > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size = 200000, > +max_join_size=2222220000000, > +keep_files_on_create=ON; > +'' > +'# LONG ' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 200000 > +SET STATEMENT sort_buffer_size = 100000 > +FOR SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 200000 > +'' > +'# MY_BOOL ' > +SHOW SESSION VARIABLES LIKE 'keep_files_on_create'; > +Variable_name Value > +keep_files_on_create ON > +SET STATEMENT keep_files_on_create=OFF > +FOR SHOW SESSION VARIABLES LIKE 'keep_files_on_create'; > +Variable_name Value > +keep_files_on_create OFF > +SHOW SESSION VARIABLES LIKE 'keep_files_on_create'; > +Variable_name Value > +keep_files_on_create ON > +'' > +'# INT/LONG ' > +SHOW SESSION VARIABLES LIKE 'myisam_repair_threads'; > +Variable_name Value > +myisam_repair_threads 1 > +SET STATEMENT myisam_repair_threads=2 > +FOR SHOW SESSION VARIABLES LIKE 'myisam_repair_threads'; > +Variable_name Value > +myisam_repair_threads 2 > +SHOW SESSION VARIABLES LIKE 'myisam_repair_threads'; > +Variable_name Value > +myisam_repair_threads 1 > +'' > +'# ULONGLONG ' > +SHOW SESSION VARIABLES LIKE 'max_join_size'; > +Variable_name Value > +max_join_size 2222220000000 > +SET STATEMENT max_join_size=2000000000000 > +FOR SHOW SESSION VARIABLES LIKE 'max_join_size'; > +Variable_name Value > +max_join_size 2000000000000 > +SHOW SESSION VARIABLES LIKE 'max_join_size'; > +Variable_name Value > +max_join_size 2222220000000 > +'' > +'#------------------Test 9-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +CREATE FUNCTION myProc (cost DECIMAL(10,2)) > +RETURNS DECIMAL(10,2) > +SQL SECURITY DEFINER > +tax: BEGIN > +DECLARE order_tax DECIMAL(10,2); > +SET order_tax = cost * .05; > +RETURN order_tax; > +END| > +'' > +'# During Execution values > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +SELECT myProc(123.45); > +myProc(123.45) > +6.17 How does this verify that SET STATEMENT had any effect? > +'' > +'# Post-STATEMENT No 1 variable value Pre-STATEMENT for No 2' > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +DROP FUNCTION myProc; How does this verify that SET STATEMENT had any effect? > +'' > +'# Post-STATEMENT No 2 variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'#------------------Test 10-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +PREPARE stmt2 > +FROM 'SELECT * FROM t1'; > +'' > +'Test No 1 Post Value & Test 2 Pre values' > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +EXECUTE stmt2; > +v1 v2 > +1 2 > +3 4 How does this verify that SET STATEMENT had any effect? Add @@binlog_format and @@sort_buffer_size to the SELECT and use different values for @@sort_buffer_size in SET STATEMENT for PREPARE and EXECUTE. Ok, I see that you do it below. Then how is this test useful? > +'' > +'# Post-STATEMENT No 2 > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +DEALLOCATE PREPARE stmt2; > +'' > +'#------------------Test 11-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +PREPARE stmt1 FROM > +'SET STATEMENT binlog_format=row FOR SELECT *,@@binlog_format FROM t1'; please also add @@sort_buffer_size to this SELECT > +'' > +'Test No 1 Post Value & Test 2 Pre values' > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +EXECUTE stmt1; > +v1 v2 @@binlog_format > +1 2 ROW > +3 4 ROW > +'' > +'# Post-STATEMENT No 2 > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +'#------------------Test 12-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +CREATE PROCEDURE p1() BEGIN > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +END| > +'' > +'Test No 1 Post Value & Test 2 Pre values' > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR Use different values here, not same as in the previous SET STATEMENT > +CALL p1(); > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +400000 2 200000 ROW 0 4444440000000 > +'' > +'# Post-STATEMENT No 2 > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +'#------------------Test 13-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'' > +CREATE PROCEDURE p2() BEGIN > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=3, > +sort_buffer_size=300000, > +binlog_format=mixed, > +keep_files_on_create=OFF, > +max_join_size=3333330000000 FOR > +CALL p1(); > +END| > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +CALL p2(); > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +400000 3 300000 MIXED 0 3333330000000 > +'' > +'# Post-STATEMENT > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +'#------------------Test 14-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'' > +CREATE PROCEDURE p3() BEGIN > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +SET STATEMENT myisam_sort_buffer_size=320000, > +myisam_repair_threads=2, > +sort_buffer_size=220022, > +binlog_format=row, > +keep_files_on_create=ON, > +max_join_size=2222220000000 FOR > +CALL p2(); > +END| > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +CALL p3(); > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +400000 2 200000 ROW 0 4444440000000 > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +400000 3 300000 MIXED 0 3333330000000 > +'' > +'# Post-STATEMENT > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +'' > +'' > +'#------------------Test 15-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'' > +CREATE PROCEDURE p4() BEGIN > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +SET STATEMENT myisam_sort_buffer_size=320000, > +myisam_repair_threads=2, > +sort_buffer_size=220022, > +binlog_format=row, > +keep_files_on_create=ON, > +max_join_size=2222220000000 FOR > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +SET STATEMENT myisam_sort_buffer_size=320000, > +myisam_repair_threads=2, > +sort_buffer_size=220022, > +binlog_format=row, > +keep_files_on_create=ON, > +max_join_size=2222220000000 FOR > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +SET STATEMENT myisam_sort_buffer_size=320000, > +myisam_repair_threads=2, > +sort_buffer_size=220022, > +binlog_format=row, > +keep_files_on_create=ON, > +max_join_size=2222220000000 FOR > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +END| > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +CALL p4(); > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +400000 2 200000 ROW 0 4444440000000 > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +320000 2 220022 ROW 1 2222220000000 > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +320000 2 220022 ROW 1 2222220000000 > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +320000 2 220022 ROW 1 2222220000000 > +'' > +'# Post-STATEMENT > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +'#------------------Test 16-----------------------#' > +'' > +'# Pre-STATEMENT variable value > +SELECT @@sql_mode; > +@@sql_mode > + > +'' > +'' > +SET STATEMENT sql_mode='ansi' FOR PREPARE stmt FROM 'SELECT "t1".* FROM t1'; > +execute stmt; > +v1 v2 > +1 2 > +3 4 Ok, that's getting interesting. Now before deallocating, try to trigger automatic reprepare. e.g. ALTER TABLE t1 ADD COLUMN v3 int; and then execute it again. also do the same with SP. I mean, set statement sql_mode=ansi create procedure this procedure uses ansi quotes. verify that the correct sql mode (ansi) is recorded in mysql.proc table. > +deallocate prepare stmt; > +'' > +'# Post-STATEMENT > +SELECT @@sql_mode; > +@@sql_mode > + > +'' > +'' > +'#------------------Test 17-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=320000, > +myisam_repair_threads=2, > +sort_buffer_size=220022, > +binlog_format=row, > +keep_files_on_create=ON, > +max_join_size=2222220000000 > +FOR SET SESSION > +myisam_sort_buffer_size=260000, > +myisam_repair_threads=3, > +sort_buffer_size=230013, > +binlog_format=row, > +keep_files_on_create=ON, > +max_join_size=2323230000000; Add a couple of other variables to SET SESSION. The point - have some variables in SET SESSION that are *not* present in SET STATEMENT. Verify that their values were correctly set. Ok, I see that you do it below. Then how is this test useful? > +'' > +'# Post-STATEMENT > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +'#------------------Test 18-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +CREATE PROCEDURE p5() BEGIN > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +SET SESSION > +myisam_sort_buffer_size=260000, > +myisam_repair_threads=3, > +sort_buffer_size=230013, > +binlog_format=row, > +keep_files_on_create=ON, > +max_join_size=2323230000000; > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +END| How is this test different from other tests above? > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +CALL p5(); > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +400000 2 200000 ROW 0 4444440000000 > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +260000 3 230013 ROW 1 2323230000000 > +'' > +'# Post-STATEMENT > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +'#------------------Test 19-----------------------#' > +SET STATEMENT max_error_count=100 FOR; > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' at line 1 > +SET STATEMENT max_error_count=100 INSERT t1 VALUES (1,2); > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'INSERT t1 VALUES (1,2)' at line 1 > +SET STATEMENT FOR INSERT INTO t1 VALUES (1,2); > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'FOR INSERT INTO t1 VALUES (1,2)' at line 1 > +SET max_error_count=100 FOR INSERT INTO t1 VALUES (1,2); > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'FOR INSERT INTO t1 VALUES (1,2)' at line 1 > +SET max_error_count=100 FOR INSERT INTO t1 VALUES (1,2); Same erroneous statement tested twice? > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'FOR INSERT INTO t1 VALUES (1,2)' at line 1 > +SET STATEMENT GLOBAL max_error_count=100 FOR INSERT INTO t1 VALUES (1,2); > +ERROR HY000: Unknown system variable 'GLOBAL' > +SET STATEMENT @@global.max_error_count=100 FOR INSERT INTO t1 VALUES (1,2); > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '@@global.max_error_count=100 FOR INSERT INTO t1 VALUES (1,2)' at line 1 > +'' > +'' > +'#------------------Test 20-----------------------#' > +SET STATEMENT connect_timeout=100 FOR INSERT INTO t1 VALUES (1,2); > +ERROR HY000: Variable 'connect_timeout' is a GLOBAL variable and should be set with SET GLOBAL > +'' > +'' > +'#------------------Test 21-----------------------#' > +SELECT @@myisam_sort_buffer_size, @@sort_buffer_size; > +@@myisam_sort_buffer_size @@sort_buffer_size > +500000 100000 > +SET STATEMENT myisam_sort_buffer_size = 700000, sort_buffer_size = 3000000 > +FOR SET STATEMENT myisam_sort_buffer_size=200000 > +FOR SELECT @@myisam_sort_buffer_size, @@sort_buffer_size; > +@@myisam_sort_buffer_size @@sort_buffer_size > +200000 100000 > +SELECT @@myisam_sort_buffer_size, @@sort_buffer_size; > +@@myisam_sort_buffer_size @@sort_buffer_size > +500000 100000 > +'' > +'' > +'#------------------Test 22-----------------------#' > +CREATE TABLE STATEMENT(a INT); > +DROP TABLE STATEMENT; > +'' > +'# Cleanup' > +DROP TABLE t1; > +DROP PROCEDURE p1; > +DROP PROCEDURE p2; > +DROP PROCEDURE p3; > +DROP PROCEDURE p4; > +DROP PROCEDURE p5; > +CREATE TABLE t1 (v1 INT, v2 INT); > +PREPARE stmt > +FROM 'SELECT * FROM t1'; > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=(select max(v1) from t1), > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +EXECUTE stmt; > +ERROR HY000: SET STATEMENT does not support using tables in the assignment variables expressions Why do you need a prepared statement to test that error? > +deallocate prepare stmt; > +drop table t1; > +set session binlog_format=mixed; > +PREPARE stmt1 FROM 'SELECT @@binlog_format'; > +execute stmt1; > +@@binlog_format > +MIXED > +set statement binlog_format=row for execute stmt1; > +@@binlog_format > +ROW > +execute stmt1; > +@@binlog_format > +MIXED > +deallocate prepare stmt1; > +set statement binlog_format=row for PREPARE stmt1 FROM 'SELECT @@binlog_format'; > +execute stmt1; > +@@binlog_format > +MIXED > +execute stmt1; > +@@binlog_format > +MIXED > +deallocate prepare stmt1; > +PREPARE stmt1 FROM 'set statement binlog_format=row for SELECT @@binlog_format'; > +execute stmt1; > +@@binlog_format > +ROW > +execute stmt1; > +@@binlog_format > +ROW > +deallocate prepare stmt1; > +set session binlog_format=default; > +set session binlog_format=mixed; > +SET STATEMENT sort_buffer_size=150000 FOR set session binlog_format=row; > +SELECT @@binlog_format; > +@@binlog_format > +ROW those are good tests, and they make many of the tests above unnecessary > +#Correctly set timestamp > +set session timestamp=4646464; > +select @@timestamp != 4646464; > +@@timestamp != 4646464 > +0 > +select @@timestamp != 4646464; > +@@timestamp != 4646464 > +0 > +#Correctly returned normal behaviour > +set session timestamp=default; > +select @@timestamp != 4646464; > +@@timestamp != 4646464 > +1 > +select @@timestamp != 4646464; > +@@timestamp != 4646464 > +1 > +#here timestamp should be set only for the statement then restored default > +set statement timestamp=4646464 for select @@timestamp; > +@@timestamp > +4646464.000000 > +set @save_tm=@@timestamp; > +select @@timestamp != 4646464; > +@@timestamp != 4646464 > +1 > +select @@timestamp != 4646464; > +@@timestamp != 4646464 > +1 > +select @@timestamp != @save_tm; > +@@timestamp != @save_tm > +1 > diff --git a/sql/set_var.cc b/sql/set_var.cc > index bae6511..20e3040 100644 > --- a/sql/set_var.cc > +++ b/sql/set_var.cc > @@ -147,7 +147,7 @@ void sys_var_end() > flags(flags_arg), show_val_type(show_val_type_arg), > guard(lock), offset(off), on_check(on_check_func), on_update(on_update_func), > deprecation_substitute(substitute), > - is_os_charset(FALSE) > + is_os_charset(FALSE), default_val(TRUE) I've asked below why is that needed. > { > /* > There is a limitation in handle_options() related to short options: > @@ -695,7 +695,8 @@ int sql_set_variables(THD *thd, List<set_var_base> *var_list) > } > > err: > - free_underlaid_joins(thd, &thd->lex->select_lex); > + if (free) > + free_underlaid_joins(thd, &thd->lex->select_lex); Ok, so in normal SET you want to free_underlaid_joins, and in SET STATEMENT you don't. Right? > DBUG_RETURN(error); > } > > diff --git a/sql/item_func.cc b/sql/item_func.cc > index 2b89aa0..651e0b1 100644 > --- a/sql/item_func.cc > +++ b/sql/item_func.cc > @@ -5556,7 +5556,7 @@ longlong Item_func_get_user_var::val_int() > tmp_var_list.push_back(new set_var_user(new Item_func_set_user_var(name, > new Item_null()))); > /* Create the variable */ > - if (sql_set_variables(thd, &tmp_var_list)) > + if (sql_set_variables(thd, &tmp_var_list, true)) Make if 'false', not 'true' here. It doesn't make any practical difference (there are no joins, the variable is set to Item_null, explicitly), but makes the logic a bit clearer (standalone SET statement = free joins, a SET that is part of another statement = don't free joins). Alternatively (and better) approach would be to remove sql_set_variables() here and use the same code that Item_func_set_user_var does. And then sql_set_variables() will be only used in SET variants, where a sysvar can be used. Here sysvars aren't possible, so sql_set_variables() is misleading. > { > thd->lex= sav_lex; > goto err; > diff --git a/sql/set_var.h b/sql/set_var.h > index e48f394..9ee914b 100644 > --- a/sql/set_var.h > +++ b/sql/set_var.h > @@ -118,6 +119,9 @@ class sys_var > */ > bool set_default(THD *thd, set_var *var); > bool update(THD *thd, set_var *var); > + void stmt_update(THD *thd) { > + on_update && on_update(this, thd, OPT_SESSION); > + } Doesn't seem to be used anywhere in the patch > > String *val_str_nolock(String *str, THD *thd, const uchar *value); > longlong val_int(bool *is_null, THD *thd, enum_var_type type, const LEX_STRING *base); > diff --git a/sql/share/errmsg-utf8.txt b/sql/share/errmsg-utf8.txt > index 233bb83..4072af4 100644 > --- a/sql/share/errmsg-utf8.txt > +++ b/sql/share/errmsg-utf8.txt > @@ -7110,3 +7110,5 @@ ER_SLAVE_SKIP_NOT_IN_GTID > eng "When using GTID, @@sql_slave_skip_counter can not be used. Instead, setting @@gtid_slave_pos explicitly can be used to skip to after a given GTID position." > ER_STATEMENT_TIMEOUT 70100 > eng "Query execution was interrupted (max_statement_time exceeded)" > +ER_SET_STATEMENT_TABLES_IS_NOT_SUPPORTED > + eng "SET STATEMENT does not support using tables in the assignment variables expressions" Better write it as ER_SUBQUERIES_NOT_SUPPORTED eng "%s does not support subqueries or stored functions." and use it for SET STATEMENT: my_error(ER_SUBQUERIES_NOT_SUPPORTED, MYF(0), "SET STATEMENT"); and also use it for KILL: - my_error(ER_NOT_SUPPORTED_YET, MYF(0), "Usage of subqueries or stored " - "function calls as part of this statement"); + my_error(ER_SUBQUERIES_NOT_SUPPORTED, MYF(0), "KILL"); > diff --git a/sql/sql_class.h b/sql/sql_class.h > index d7bbfc3..0f6b39e 100644 > --- a/sql/sql_class.h > +++ b/sql/sql_class.h > @@ -3711,6 +3711,10 @@ class THD :public Statement, > void rgi_unlock_temporary_tables(); > bool rgi_have_temporary_tables(); > public: > + inline MEM_ROOT *get_execution_mem_root() > + { > + return &main_mem_root; > + } Doesn't seem to be used anywhere in this patch > /* > Flag, mutex and condition for a thread to wait for a signal from another > thread. > diff --git a/sql/sql_lex.h b/sql/sql_lex.h > index 5e9c7b9..03861d1 100644 > --- a/sql/sql_lex.h > +++ b/sql/sql_lex.h > @@ -2379,6 +2381,8 @@ struct LEX: public Query_tables_list > required a local context, the parser pops the top-most context. > */ > List<Name_resolution_context> context_stack; > + /* true if SET STATEMENT ... FOR ... statement is use, false otherwise */ > + bool set_statement; This doesn't seem to be needed, I've checked all places where set_statement is used below. > > SQL_I_List<ORDER> proc_list; > SQL_I_List<TABLE_LIST> auxiliary_table_list, save_list; > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index d8906b2..eff0e08 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -2637,6 +2637,98 @@ static bool do_execute_sp(THD *thd, sp_head *sp) > thd->get_binlog_format(&orig_binlog_format, > &orig_current_stmt_binlog_format); > > + if (lex->set_statement && !lex->stmt_var_list.is_empty()) Why do you check lex->set_statement? I'd think that if (!lex->stmt_var_list.is_empty()) is quite enough. > + { > + DBUG_PRINT("info", ("SET STATEMENT %d vars", lex->stmt_var_list.elements)); > + > + lex->old_var_list.empty(); > + List_iterator_fast<set_var_base> it(lex->stmt_var_list); > + set_var_base *var; > + while ((var=it++)) > + { > + DBUG_ASSERT(var->is_system()); > + set_var *o= NULL, *v= (set_var*)var; > + if (v->var->is_default()) > + o= new set_var(v->type, v->var, &v->base, NULL); Why do you treat default values differently? > + else > + { > + switch (v->var->option.var_type) > + { > + case GET_BOOL: > + case GET_INT: > + case GET_LONG: > + case GET_LL: > + { > + bool null_value; > + longlong val= v->var->val_int(&null_value, thd, v->type, &v->base); > + o= new set_var(v->type, v->var, &v->base, > + (null_value ? > + (Item *)new Item_null() : > + (Item *)new Item_int(val))); > + } > + break; > + case GET_UINT: > + case GET_ULONG: > + case GET_ULL: > + { > + bool null_value; > + ulonglong val= v->var->val_int(&null_value, thd, v->type, &v->base); > + o= new set_var(v->type, v->var, &v->base, > + (null_value ? > + (Item *)new Item_null() : > + (Item *)new Item_uint(val))); > + } > + break; > + case GET_DOUBLE: > + { > + bool null_value; > + double val= v->var->val_real(&null_value, thd, v->type, &v->base); > + o= new set_var(v->type, v->var, &v->base, > + (null_value ? > + (Item *)new Item_null() : > + (Item *)new Item_float(val, 1))); > + } > + break; > + default: > + case GET_NO_ARG: > + case GET_DISABLED: > + DBUG_ASSERT(0); > + case 0: > + case GET_FLAGSET: > + case GET_ASK_ADDR: > + case GET_TYPE_MASK: > + case GET_ENUM: > + case GET_SET: > + case GET_STR: > + case GET_STR_ALLOC: > + { > + char buff[STRING_BUFFER_USUAL_SIZE]; > + String tmp(buff, sizeof(buff), v->var->charset(thd)),*val; > + val= v->var->val_str(&tmp, thd, v->type, &v->base); > + if (val) > + { > + Item_string *str= new Item_string(v->var->charset(thd)); > + str->set_str_with_copy(val->ptr(), val->length()); > + o= new set_var(v->type, v->var, &v->base, str); > + } > + else > + o= new set_var(v->type, v->var, &v->base, new Item_null()); > + } > + break; > + } > + } > + DBUG_ASSERT(o); > + lex->old_var_list.push_back(o); > + } > + if (thd->is_error() || > + (res= sql_set_variables(thd, &lex->stmt_var_list, false))) > + { > + if (!thd->is_error()) > + my_error(ER_WRONG_ARGUMENTS, MYF(0), "SET"); > + goto error; > + } > + } > + > /* > Force statement logging for DDL commands to allow us to update > privilege, system or statistic tables directly without the updates > @@ -2812,6 +2904,12 @@ static bool do_execute_sp(THD *thd, sp_head *sp) > } > case SQLCOM_EXECUTE: > { > + /* > + Clean up free_list here from Items used in setting statement variables > + to present clear item list to Prepared_statement::execute_loop which > + need it empty because switch arena (and so item list) on statement arena. > + */ > + //free_items(thd->free_list); thd->free_list= NULL; Eh? Forgot to remove it? > mysql_sql_stmt_execute(thd); > break; > } > diff --git a/sql/sql_plugin.cc b/sql/sql_plugin.cc > index 9ae3d79..4b54989 100644 > --- a/sql/sql_plugin.cc > +++ b/sql/sql_plugin.cc > @@ -4078,3 +4078,48 @@ static void restore_ptr_backup(uint n, st_ptr_backup *backup) > (backup++)->restore(); > } > > + > +/** > + Create deep copy of system_variables instance. > +*/ > +struct system_variables * > +copy_system_variables(const struct system_variables *src) Doesn't seem to be used anywhere in the patch > +{ > + struct system_variables *dst; > + > + DBUG_ASSERT(src); > + > + dst= (struct system_variables *) > + sql_alloc(sizeof(struct system_variables)); > + if (!dst) > + return NULL; > + *dst= *src; > + > + if (dst->dynamic_variables_ptr) > + { > + if (!(dst->dynamic_variables_ptr= > + (char *)my_malloc(dst->dynamic_variables_size, MYF(MY_WME | MY_FAE)))) > + return NULL; > + memcpy(dst->dynamic_variables_ptr, > + src->dynamic_variables_ptr, > + src->dynamic_variables_size); > + } > + > + mysql_mutex_lock(&LOCK_plugin); > + dst->table_plugin= > + intern_plugin_lock(NULL, src->table_plugin); > + mysql_mutex_unlock(&LOCK_plugin); > + > + return dst; > +} > + > +void free_system_variables(struct system_variables *v) Doesn't seem to be used anywhere in the patch > +{ > + DBUG_ASSERT(v); > + > + mysql_mutex_lock(&LOCK_plugin); > + intern_plugin_unlock(NULL, v->table_plugin); > + mysql_mutex_unlock(&LOCK_plugin); > + > + my_free(v->dynamic_variables_ptr); > +} > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index e7fcdfb..d140597 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -14416,8 +14420,52 @@ set: > } > start_option_value_list > {} > + | SET STATEMENT_SYM > + { > + LEX *lex= Lex; > + mysql_init_select(lex); > + lex->sql_command= SQLCOM_SET_OPTION; > + /* Don't clear var_list in the case of recursive statement */ What's "recursive set statement"? SET STATEMENT a=1 FOR SET STATEMENT ? I'd simply return a syntax error here. Like MYSQL_YYABORT_UNLESS(lex->stmt_var_list.is_empty()); > + if (!lex->set_statement) > + { > + lex->var_list.empty(); > + lex->stmt_var_list.empty(); Why here, and not in lex_start() ? > + } > + lex->autocommit= 0; > + lex->set_statement= true; > + sp_head *sp= lex->sphead; > + if (sp && !sp->is_invoked()) > + { > + sp->m_param_begin= ((yychar == YYEMPTY) ? YYLIP->get_ptr() : YYLIP->get_tok_start()); > + sp->m_param_end= ((yychar == YYEMPTY) ? YYLIP->get_ptr() : YYLIP->get_tok_end()); what's that? > + } > + } > + set_stmt_option_value_following_option_type_list > + { > + LEX *lex= Lex; > + if (lex->query_tables) > + { > + my_error(ER_SET_STATEMENT_TABLES_IS_NOT_SUPPORTED, MYF(0)); why? what about stored functions that use tables internally? > + MYSQL_YYABORT; > + } > + //if (lex->set_statement && lex->stmt_var_list.is_empty()) > + { > + lex->stmt_var_list= lex->var_list; > + lex->var_list.empty(); > + } > + } > + FOR_SYM statement > + {} > ; > > +set_stmt_option_value_following_option_type_list: > + /* > + Only system variables can be used here. If this condition is changed > + please check careful code under lex->option_type == OPT_STATEMENT > + condition on wrong type casts. > + */ > + option_value_following_option_type > + | set_stmt_option_value_following_option_type_list ',' option_value_following_option_type > > // Start of option value list > start_option_value_list: > @@ -14522,6 +14570,14 @@ option_value_following_option_type: > { > LEX *lex= Lex; > > + /* > + Ignore SET STATEMENT variables list on slaves because system > + variables are not replicated except certain variables set the > + values of whose are written to binlog event header and nothing > + additional is required to set them. > + */ > + if (!(thd->slave_thread && lex->set_statement)) > + { Hmm, okay. Alternatively, you can simply make sure that set statement is not written into binlog. Or don't execute assignments in the slave thread. > if ($1.var && $1.var != trg_new_row_fake_var) > { > /* It is a system variable. */ Regards, Sergei

2 3

Re: [Maria-developers] [Commits] Rev 4445: MDEV-6878: Use of uninitialized saved_primary_key in Mrr_ordered_index_reader::resume_read() in file:///home/psergey/dev2/10.0/
by Jeremy Cole 20 Oct '14

20 Oct '14

Sergey, Nice test case. Would you mind cleaning up the end-space in your patch? Also, is there a reason to put the entire body of Mrr_ordered_index_reader::resume_read in the if() instead of an early return? Regards, Jeremy On Thu, Oct 16, 2014 at 11:58 AM, Sergey Petrunya <psergey(a)askmonty.org> wrote: > At file:///home/psergey/dev2/10.0/ > > ------------------------------------------------------------ > revno: 4445 > revision-id: psergey(a)askmonty.org-20141016135713-md92qr4g9fzhitxh > parent: bar(a)mnogosearch.org-20141013083155-j3j31hqhfc8i95y1 > committer: Sergey Petrunya <psergey(a)askmonty.org> > branch nick: 10.0 > timestamp: Thu 2014-10-16 17:57:13 +0400 > message: > MDEV-6878: Use of uninitialized saved_primary_key in > Mrr_ordered_index_reader::resume_read() > > - Make Mrr_ordered_index_reader::resume_read() restore index position > only if it was saved before with > Mrr_ordered_index_reader::interrupt_read(). > === modified file 'mysql-test/r/innodb_mrr_cpk.result' > --- a/mysql-test/r/innodb_mrr_cpk.result 2013-09-26 18:20:15 +0000 > +++ b/mysql-test/r/innodb_mrr_cpk.result 2014-10-16 13:57:13 +0000 > @@ -194,3 +194,41 @@ id select_type table type possible_keys > 2 DERIVED t2 ALL NULL NULL NULL NULL # > set join_cache_level= @tmp_mdev5037; > drop table t0,t1,t2; > +# > +# MDEV-6878: Use of uninitialized saved_primary_key in > Mrr_ordered_index_reader::resume_read() > +# > +create table t0(a int); > +insert into t0 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); > +create table t1 ( > +pk varchar(32) character set utf8 primary key, > +kp1 char(32) not null, > +col1 varchar(32), > +key (kp1) > +) engine=innodb; > +insert into t1 > +select > +concat('pk-', 1000 +A.a), > +concat('kp1-', 1000 +A.a), > +concat('val-', 1000 +A.a) > +from test.t0 A ; > +create table t2 as select kp1 as a from t1; > +set join_cache_level=8; > +set optimizer_switch='mrr=on,mrr_sort_keys=on'; > +explain > +select * from t2 straight_join t1 force index(kp1) where t1.kp1=t2.a; > +id select_type table type possible_keys key key_len > ref rows Extra > +1 SIMPLE t2 ALL NULL NULL NULL NULL 10 > +1 SIMPLE t1 ref kp1 kp1 32 test.t2.a 1 > Using join buffer (flat, BKAH join); Key-ordered Rowid-ordered scan > +select * from t2 straight_join t1 force index(kp1) where t1.kp1=t2.a; > +a pk kp1 col1 > +kp1-1000 pk-1000 kp1-1000 val-1000 > +kp1-1001 pk-1001 kp1-1001 val-1001 > +kp1-1002 pk-1002 kp1-1002 val-1002 > +kp1-1003 pk-1003 kp1-1003 val-1003 > +kp1-1004 pk-1004 kp1-1004 val-1004 > +kp1-1005 pk-1005 kp1-1005 val-1005 > +kp1-1006 pk-1006 kp1-1006 val-1006 > +kp1-1007 pk-1007 kp1-1007 val-1007 > +kp1-1008 pk-1008 kp1-1008 val-1008 > +kp1-1009 pk-1009 kp1-1009 val-1009 > +drop table t0,t1,t2; > > === modified file 'mysql-test/t/innodb_mrr_cpk.test' > --- a/mysql-test/t/innodb_mrr_cpk.test 2013-09-26 18:20:15 +0000 > +++ b/mysql-test/t/innodb_mrr_cpk.test 2014-10-16 13:57:13 +0000 > @@ -192,3 +192,35 @@ explain SELECT 1 FROM (SELECT url, id FR > set join_cache_level= @tmp_mdev5037; > > drop table t0,t1,t2; > + > +--echo # > +--echo # MDEV-6878: Use of uninitialized saved_primary_key in > Mrr_ordered_index_reader::resume_read() > +--echo # > +create table t0(a int); > +insert into t0 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); > + > +create table t1 ( > + pk varchar(32) character set utf8 primary key, > + kp1 char(32) not null, > + col1 varchar(32), > + key (kp1) > +) engine=innodb; > + > +insert into t1 > +select > + concat('pk-', 1000 +A.a), > + concat('kp1-', 1000 +A.a), > + concat('val-', 1000 +A.a) > +from test.t0 A ; > + > +create table t2 as select kp1 as a from t1; > + > +set join_cache_level=8; > +set optimizer_switch='mrr=on,mrr_sort_keys=on'; > +explain > +select * from t2 straight_join t1 force index(kp1) where t1.kp1=t2.a; > +select * from t2 straight_join t1 force index(kp1) where t1.kp1=t2.a; > + > +drop table t0,t1,t2; > + > + > > === modified file 'sql/multi_range_read.cc' > --- a/sql/multi_range_read.cc 2014-06-06 17:28:42 +0000 > +++ b/sql/multi_range_read.cc 2014-10-16 13:57:13 +0000 > @@ -426,6 +426,7 @@ bool Mrr_ordered_index_reader::set_inter > *space_start += key_len; > > have_saved_rowid= FALSE; > + read_was_interrupted= FALSE; > return FALSE; > } > > @@ -434,6 +435,7 @@ void Mrr_ordered_index_reader::set_no_in > support_scan_interruptions= FALSE; > saved_key_tuple= saved_rowid= saved_primary_key= NULL; /* safety */ > have_saved_rowid= FALSE; > + read_was_interrupted= FALSE; > } > > void Mrr_ordered_index_reader::interrupt_read() > @@ -451,6 +453,7 @@ void Mrr_ordered_index_reader::interrupt > &table->key_info[table->s->primary_key], > table->key_info[table->s->primary_key].key_length); > } > + read_was_interrupted= TRUE; > > /* Save the last rowid */ > memcpy(saved_rowid, file->ref, file->ref_length); > @@ -468,14 +471,17 @@ void Mrr_ordered_index_reader::position( > void Mrr_ordered_index_reader::resume_read() > { > TABLE *table= file->get_table(); > - KEY *used_index= &table->key_info[file->active_index]; > - key_restore(table->record[0], saved_key_tuple, > - used_index, used_index->key_length); > - if (saved_primary_key) > + if (read_was_interrupted) > { > - key_restore(table->record[0], saved_primary_key, > - &table->key_info[table->s->primary_key], > - table->key_info[table->s->primary_key].key_length); > + KEY *used_index= &table->key_info[file->active_index]; > + key_restore(table->record[0], saved_key_tuple, > + used_index, used_index->key_length); > + if (saved_primary_key) > + { > + key_restore(table->record[0], saved_primary_key, > + &table->key_info[table->s->primary_key], > + table->key_info[table->s->primary_key].key_length); > + } > } > } > > @@ -557,8 +563,7 @@ int Mrr_ordered_index_reader::init(handl > is_mrr_assoc= !MY_TEST(mode & HA_MRR_NO_ASSOCIATION); > mrr_funcs= *seq_funcs; > source_exhausted= FALSE; > - if (support_scan_interruptions) > - bzero(saved_key_tuple, key_info->key_length); > + read_was_interrupted= false; > have_saved_rowid= FALSE; > return 0; > } > > === modified file 'sql/multi_range_read.h' > --- a/sql/multi_range_read.h 2013-09-26 18:20:15 +0000 > +++ b/sql/multi_range_read.h 2014-10-16 13:57:13 +0000 > @@ -339,6 +339,12 @@ class Mrr_ordered_index_reader : public > > uchar *saved_key_tuple; /* Saved current key tuple */ > uchar *saved_primary_key; /* Saved current primary key tuple */ > + > + /* > + TRUE<=> saved_key_tuple (and saved_primary_key when applicable) have > + valid values. > + */ > + bool read_was_interrupted; > > static int compare_keys(void* arg, uchar* key1, uchar* key2); > static int compare_keys_reverse(void* arg, uchar* key1, uchar* key2); > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits >

2 1

Re: [Maria-developers] MariaDB 10.0.14 got rid of replication safety warning without warning in release notes?
by Pavel Ivanov 20 Oct '14

20 Oct '14

> Pavel> Which key is "the first duplicate key". > > The first key that has a duplicate value in the order of keys in the KEY > structure that MariaDB provides for the storage engine. I'm definitely not that familiar with the relevant MariaDB code, but I didn't find KEY structure to be passed to any insert-related function. Can you elaborate (preferably with code references) how this restriction is supposed to work? > This is the same order that keys are presented in SHOW CREATE TABLE Why is it safe to assume that keys have the same order on master and on slave? > To help engine writes we are constantly adding new test in the storage > engine test suite, to make it easier for storage engine vendors to > test that their storage engine follows the interface. > We have now added a test to check for the correct behaviour for the > case of INSERT ... ON DUPLICATE KEY. This is very interesting. Can you say which test it is and what exactly does it test? Is it present in 10.0.14 or it's only at the latest 10.0 branch? > Do you have or use a storage engine that doesn't provide the expected value > for the conflicting key ? I don't have a specific example of storage engine that detects duplicate key in different order, but I'm not convinced and don't have hard proof that such engine cannot exist and that InnoDB itself is not susceptible to this problem. I can easily imagine though that even given the particular order of keys in table definition the key reported as having duplicate can depend on the way storage engine stores those keys. BTW, can you point to MySQL documentation and/or comment in the code that indicates this requirement to storage engine implementation? > Pavel> On the related note Release Notes mention the error log flood > Pavel> protection. Why it's not behind flag? How should I disable the feature > Pavel> and get back to the old behavior? > > We already had in MariaDB flood protection for some error > messages. Can you say exactly which error messages are suppressed already? Is there a documentation link about that? > I am happy to look into adding a flag that is usable for any kind of > suppressed messages. > > A couple of ways to do this: > > a) Not suppress anything if you give --log-warnings a value > 2 > b) make LIMIT_UNSAFE_WARNING_ACTIVATION_TIMEOUT a configurable > variable, where 0 would mean no suppression. > > Which of the above, or both, would you prefer? > Or do you have any other suggestion of how to do this? Reading documentation on --log-warnings I don't see how it can be related to the error suppression. So I think making LIMIT_UNSAFE_WARNING_ACTIVATION_TIMEOUT configurable would be much better, though it should apply to all possible error suppressions. Thank you, Pavel On Fri, Oct 17, 2014 at 1:43 PM, Michael Widenius <michael.widenius(a)gmail.com> wrote: > > Hi! > > First sorry for the delayed reply; I am at a conference in Oxford and > didn't have an reliable internet connection until now. > >>>>>> "Pavel" == Pavel Ivanov <pivanof(a)google.com> writes: > > Pavel> Hi, > Pavel> I see that in MariaDB 10.0.14 Michael disabled replication safety > Pavel> warning for the case of INSERT ON DUPLICATE KEY UPDATE with several > Pavel> unique keys with a comment "We should assume that the store engine > Pavel> will report the first duplicate key for this case." > Pavel> http://bazaar.launchpad.net/~maria-captains/maria/10.0/revision/4400.1.3 > > Pavel> What does that mean? > > That we don't anymore give a warning for things that can't happen in > the distributed MariaDB server. > > Pabel> Which key is "the first duplicate key". > > The first key that has a duplicate value in the order of keys in the KEY > structure that MariaDB provides for the storage engine. > > This is the same order that keys are presented in SHOW CREATE TABLE > > Pavel> and why > Pavel> MariaDB relies on an engine behavior which AFAIK is not guaranteed? > > This is guaranteed for all storage engines shipped with MariaDB and > should also be guaranteed for any storage engine that wants to follow > the MariaDB and even the original MySQL storage engine interface. > > The change is also in line with how I designed the part of the storage > engine interface that gives the SQL layer the information about which > key that was conflicted. As we use this for things like INSERT ... ON > DUPLICATE KEY, it's critical that all storage engine provides the same > information in case of conflicts. > > To help engine writes we are constantly adding new test in the storage > engine test suite, to make it easier for storage engine vendors to > test that their storage engine follows the interface. > We have now added a test to check for the correct behaviour for the > case of INSERT ... ON DUPLICATE KEY. > > What is important when it comes to the storage engine interface is > that in all possible cases, a query should work identically against 2 > different storage engines. > > The only 'allowed difference's that I can think of for the moment are: > > - Different storage engines may retrieve rows in different order. > - In case of errors, changes to non transactional tables will not roll > back. > - Automatically generated key values may be different between storage engines. > (but preferably they should not be that). However when things are > replicated the slave, it should get the same values as the master. > > In the case of INSERT ... ON DUPLICATE KEY, it was clear that someone > had broke the storage engine interface, for no good reason. Instead > of users having to depend on 'unsafe, different and undocumented > behavior' for which row as updated, it's better to fix this issue once > and for all, especially as fixing this is trivial. > > Do you have or use a storage engine that doesn't provide the expected value > for the conflicting key ? > > If yes, there is a couple of ways to fix this. I am happy to talk with > you to find a solution that will satisfy your needs and ours. > > I underatand that in many cases it's not critical which unique key the > storage engine reports as duplicate. However in some cases, it's > critical to get the first key. > > I could add a new extra call to inform the engine that we now require > the first duplicate key and not just any conflicting key. This would > be used for example for INSERT ... ON DUPLICATE KEY ... > > I would prefer to have the check for which key was duplicated in the > storage engine. However, if critically needed, and assuming there is more > than one storage engine that needs this, we could add a check on the > handler level that will do a lookup on all unique keys to find the > first duplicate one. > > What solution would you prefer? > > Pavel> And why this serious change in behavior wasn't mentioned in the > Pavel> Release Notes? > > I didn't see it as a a serious change as it's only about removing a > warning that has no value for any normal MariaDB user (who only use > storage engines supplied with MariadB) and which actually caused > serious problems for some MariaDB users. > > That said, I agree it was a mistake from my part to not notice that > this was missing in the release notes. I will try to do better in the > future and check that the release notes correctly reflects any changes > that I have done. > > I appologize for missing this in this case. > > Pavel> On the related note Release Notes mention the error log flood > Pavel> protection. Why it's not behind flag? How should I disable the feature > Pavel> and get back to the old behavior? > > We already had in MariaDB flood protection for some error > messages. What I did was to make it more general and extend it to also > include other error messages, like the one we are discussing. As > there wasn't a flag for the old suppression, I didn't see a need for > adding a flag for the new suppression. > > I am happy to look into adding a flag that is usable for any kind of > suppressed messages. > > A couple of ways to do this: > > a) Not suppress anything if you give --log-warnings a value > 2 > b) make LIMIT_UNSAFE_WARNING_ACTIVATION_TIMEOUT a configurable > variable, where 0 would mean no suppression. > > Which of the above, or both, would you prefer? > Or do you have any other suggestion of how to do this? > > For 10.1 we will probably add a mechanism where you can permanently > suppress any specific error message to be written to the error log. > (They would still be given to the client). > > Any suggestion or wishes regarding this? > > Regards, > Monty

1 0

[Maria-developers] Igor please review: [Commits] Rev 4446: MDEV-6879: Dereference of NULL primary_file->table in DsMrr_impl::get_disk_sweep_mrr_cost()
by Sergey Petrunia 20 Oct '14

20 Oct '14

Hello Igor, Could you please review the below: At file:///home/psergey/dev2/10.0-cp2/ ------------------------------------------------------------ revno: 4446 revision-id: psergey at askmonty.org-20141020175620-vodbkx9ugmdhh8c1 parent: psergey at askmonty.org-20141016135713-md92qr4g9fzhitxh committer: Sergey Petrunya <psergey at askmonty.org> branch nick: 10.0-cp2 timestamp: Mon 2014-10-20 21:56:20 +0400 message: MDEV-6879: Dereference of NULL primary_file->table in DsMrr_impl::get_disk_sweep_mrr_cost() (Attempt #2) - Don't attempt to use BKA for materialized derived tables. The table is neither filled nor fully opened yet, so attempt to call handler->multi_range_read_info() causes crash. === modified file 'mysql-test/r/derived_opt.result' --- a/mysql-test/r/derived_opt.result 2013-06-18 15:21:00 +0000 +++ b/mysql-test/r/derived_opt.result 2014-10-20 17:56:20 +0000 @@ -352,4 +352,33 @@ pk pk 72 72 80 80 drop table t1, t2, t3, t4; +# +# MDEV-6879: Dereference of NULL primary_file->table in DsMrr_impl::get_disk_sweep_mrr_cost() +# +create table t1(a int); +insert into t1 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); +create table t2 (a int, b int, c text); +insert into t2 +select +A.a + B.a* 10, +A.a + B.a* 10, +'blob-data' +from t1 A, t1 B; +set @tmp_jcl= @@join_cache_level; +set @tmp_os= @@optimizer_switch; +set join_cache_level=6; +set @@optimizer_switch='derived_merge=on,derived_with_keys=on,mrr=on'; +explain +select * from +t1 join +(select * from t2 order by a limit 1000) as D1 +where +D1.a= t1.a; +id select_type table type possible_keys key key_len ref rows Extra +1 PRIMARY t1 ALL NULL NULL NULL NULL 10 Using where +1 PRIMARY <derived2> ref key0 key0 5 test.t1.a 10 +2 DERIVED t2 ALL NULL NULL NULL NULL 100 Using filesort +set join_cache_level=@tmp_jcl; +set optimizer_switch=@tmp_os; +drop table t1, t2; set optimizer_switch=@exit_optimizer_switch; === modified file 'mysql-test/t/derived_opt.test' --- a/mysql-test/t/derived_opt.test 2012-11-19 16:17:46 +0000 +++ b/mysql-test/t/derived_opt.test 2014-10-20 17:56:20 +0000 @@ -272,5 +272,34 @@ limit 10; drop table t1, t2, t3, t4; +--echo # +--echo # MDEV-6879: Dereference of NULL primary_file->table in DsMrr_impl::get_disk_sweep_mrr_cost() +--echo # +create table t1(a int); +insert into t1 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); + +create table t2 (a int, b int, c text); +insert into t2 +select + A.a + B.a* 10, + A.a + B.a* 10, + 'blob-data' +from t1 A, t1 B; + +set @tmp_jcl= @@join_cache_level; +set @tmp_os= @@optimizer_switch; +set join_cache_level=6; +set @@optimizer_switch='derived_merge=on,derived_with_keys=on,mrr=on'; +explain +select * from + t1 join + (select * from t2 order by a limit 1000) as D1 +where + D1.a= t1.a; + +set join_cache_level=@tmp_jcl; +set optimizer_switch=@tmp_os; +drop table t1, t2; + # The following command must be the last one the file set optimizer_switch=@exit_optimizer_switch; === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2014-10-09 08:30:11 +0000 +++ b/sql/sql_select.cc 2014-10-20 17:56:20 +0000 @@ -10550,6 +10550,19 @@ uint check_join_cache_usage(JOIN_TAB *ta } /* + Don't use BKA for materialized tables. We could actually have a + meaningful use of BKA when linked join buffers are used. + + The problem is, the temp.table is not filled (actually not even opened + properly) yet, and this doesn't let us call + handler->multi_range_read_info(). It is possible to come up with + estimates, etc. without acessing the table, but it seems not to worth the + effort now. + */ + if (tab->table->pos_in_table_list->is_materialized_derived()) + no_bka_cache= true; + + /* Don't use join buffering if we're dictated not to by no_jbuf_after (This is not meaningfully used currently) */ @@ -10615,7 +10628,7 @@ uint check_join_cache_usage(JOIN_TAB *ta if (tab->ref.is_access_triggered()) goto no_join_cache; - if (!tab->is_ref_for_hash_join()) + if (!tab->is_ref_for_hash_join() && !no_bka_cache) { flags= HA_MRR_NO_NULL_ENDPOINTS | HA_MRR_SINGLE_POINT; if (tab->table->covering_keys.is_set(tab->ref.key)) BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] review MDEV 6631
by Roberto Spadim 18 Oct '14

18 Oct '14

hi guys could review MDEV-6631? there's a pull request, maybe should include some test files, but I don't know how to implement profiler check with mysql test tools -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

Re: [Maria-developers] MariaDB 10.0.14 got rid of replication safety warning without warning in release notes?
by Pavel Ivanov 18 Oct '14

18 Oct '14

As I have reports of people not seeing this email on maria-developers@ list (probably because Monty sent it from the wrong address) I'm forwarding it to the list. I'll respond to it in detail later. On Fri, Oct 17, 2014 at 1:43 PM, Michael Widenius <michael.widenius(a)gmail.com> wrote: > > Hi! > > First sorry for the delayed reply; I am at a conference in Oxford and > didn't have an reliable internet connection until now. > >>>>>> "Pavel" == Pavel Ivanov <pivanof(a)google.com> writes: > > Pavel> Hi, > Pavel> I see that in MariaDB 10.0.14 Michael disabled replication safety > Pavel> warning for the case of INSERT ON DUPLICATE KEY UPDATE with several > Pavel> unique keys with a comment "We should assume that the store engine > Pavel> will report the first duplicate key for this case." > Pavel> http://bazaar.launchpad.net/~maria-captains/maria/10.0/revision/4400.1.3 > > Pavel> What does that mean? > > That we don't anymore give a warning for things that can't happen in > the distributed MariaDB server. > > Pabel> Which key is "the first duplicate key". > > The first key that has a duplicate value in the order of keys in the KEY > structure that MariaDB provides for the storage engine. > > This is the same order that keys are presented in SHOW CREATE TABLE > > Pavel> and why > Pavel> MariaDB relies on an engine behavior which AFAIK is not guaranteed? > > This is guaranteed for all storage engines shipped with MariaDB and > should also be guaranteed for any storage engine that wants to follow > the MariaDB and even the original MySQL storage engine interface. > > The change is also in line with how I designed the part of the storage > engine interface that gives the SQL layer the information about which > key that was conflicted. As we use this for things like INSERT ... ON > DUPLICATE KEY, it's critical that all storage engine provides the same > information in case of conflicts. > > To help engine writes we are constantly adding new test in the storage > engine test suite, to make it easier for storage engine vendors to > test that their storage engine follows the interface. > We have now added a test to check for the correct behaviour for the > case of INSERT ... ON DUPLICATE KEY. > > What is important when it comes to the storage engine interface is > that in all possible cases, a query should work identically against 2 > different storage engines. > > The only 'allowed difference's that I can think of for the moment are: > > - Different storage engines may retrieve rows in different order. > - In case of errors, changes to non transactional tables will not roll > back. > - Automatically generated key values may be different between storage engines. > (but preferably they should not be that). However when things are > replicated the slave, it should get the same values as the master. > > In the case of INSERT ... ON DUPLICATE KEY, it was clear that someone > had broke the storage engine interface, for no good reason. Instead > of users having to depend on 'unsafe, different and undocumented > behavior' for which row as updated, it's better to fix this issue once > and for all, especially as fixing this is trivial. > > Do you have or use a storage engine that doesn't provide the expected value > for the conflicting key ? > > If yes, there is a couple of ways to fix this. I am happy to talk with > you to find a solution that will satisfy your needs and ours. > > I underatand that in many cases it's not critical which unique key the > storage engine reports as duplicate. However in some cases, it's > critical to get the first key. > > I could add a new extra call to inform the engine that we now require > the first duplicate key and not just any conflicting key. This would > be used for example for INSERT ... ON DUPLICATE KEY ... > > I would prefer to have the check for which key was duplicated in the > storage engine. However, if critically needed, and assuming there is more > than one storage engine that needs this, we could add a check on the > handler level that will do a lookup on all unique keys to find the > first duplicate one. > > What solution would you prefer? > > Pavel> And why this serious change in behavior wasn't mentioned in the > Pavel> Release Notes? > > I didn't see it as a a serious change as it's only about removing a > warning that has no value for any normal MariaDB user (who only use > storage engines supplied with MariadB) and which actually caused > serious problems for some MariaDB users. > > That said, I agree it was a mistake from my part to not notice that > this was missing in the release notes. I will try to do better in the > future and check that the release notes correctly reflects any changes > that I have done. > > I appologize for missing this in this case. > > Pavel> On the related note Release Notes mention the error log flood > Pavel> protection. Why it's not behind flag? How should I disable the feature > Pavel> and get back to the old behavior? > > We already had in MariaDB flood protection for some error > messages. What I did was to make it more general and extend it to also > include other error messages, like the one we are discussing. As > there wasn't a flag for the old suppression, I didn't see a need for > adding a flag for the new suppression. > > I am happy to look into adding a flag that is usable for any kind of > suppressed messages. > > A couple of ways to do this: > > a) Not suppress anything if you give --log-warnings a value > 2 > b) make LIMIT_UNSAFE_WARNING_ACTIVATION_TIMEOUT a configurable > variable, where 0 would mean no suppression. > > Which of the above, or both, would you prefer? > Or do you have any other suggestion of how to do this? > > For 10.1 we will probably add a mechanism where you can permanently > suppress any specific error message to be written to the error log. > (They would still be given to the client). > > Any suggestion or wishes regarding this? > > Regards, > Monty

1 0

Re: [Maria-developers] [Commits] Rev 3900: Fix for test failures on 64-bit platform. in lp:~maria-captains/maria/10.0-galera-6593
by Kristian Nielsen 18 Oct '14

18 Oct '14

Nirbhay Choubey <nirbhay(a)mariadb.com> writes: > message: > Fix for test failures on 64-bit platform. > === modified file 'sql/rpl_mi.cc' > --- a/sql/rpl_mi.cc 2014-10-10 23:06:40 +0000 > +++ b/sql/rpl_mi.cc 2014-10-17 02:08:55 +0000 > @@ -1466,7 +1466,7 @@ > { > for (int i= DO_DOMAIN_IDS; i <= IGNORE_DOMAIN_IDS; i ++) > { > - my_init_dynamic_array(&m_domain_ids[i], sizeof(uint32), 16, 16, MYF(0)); > + my_init_dynamic_array(&m_domain_ids[i], sizeof(ulong), 16, 16, MYF(0)); > } > } > Why do you need to use ulong? Domain ids should always be uint32, also on 64-bit. What is the real problem? Eg. why is there a test failure? I'd expect that the right fix was to convert to uint32 in the places where a domain_id might come into the code as ulong... > @@ -1483,7 +1483,7 @@ > domain ids list. DO_DOMAIN_IDS list is only looked-up is both (do & ignore) > list are non-empty. > */ > -void Domain_id_filter::do_filter(uint32 domain_id) > +void Domain_id_filter::do_filter(ulong domain_id) > { > DYNAMIC_ARRAY *do_domain_ids= &m_domain_ids[DO_DOMAIN_IDS]; > DYNAMIC_ARRAY *ignore_domain_ids= &m_domain_ids[IGNORE_DOMAIN_IDS]; > @@ -1491,21 +1491,21 @@ > if (do_domain_ids->elements > 0) > { > if (likely(do_domain_ids->elements == 1)) > - m_filter= ((* (uint32*) dynamic_array_ptr(do_domain_ids, 0)) > + m_filter= ((* (ulong *) dynamic_array_ptr(do_domain_ids, 0)) > != domain_id); > else > - m_filter= (bsearch((const uint32 *) &domain_id, do_domain_ids->buffer, > - do_domain_ids->elements, sizeof(uint32), > + m_filter= (bsearch((const ulong *) &domain_id, do_domain_ids->buffer, > + do_domain_ids->elements, sizeof(ulong), Why is a cast needed here? If it's because of const, could you avoid the cast just by declaring domain_id const in the argument of the function? Thanks, - Kristian.

2 1

[Maria-developers] doubts about show ___ any information table plugin
by Roberto Spadim 17 Oct '14

17 Oct '14

hi guys ! :) i was reading this: https://mariadb.com/kb/en/mariadb/development/mariadb-internals-documentati… and i'm with some doubts... automatically parser translate "_" to space? for example table A_B, i could use SHOW A B ? about the FLUSH what it does? any example of what it should do? like... i know we can reset some information counters, it's a 'shortcut' to avoid something like "SET pluginvariable=0"? if yes, maybe we could add some example at KB page, i'm with doubts, maybe a dba without knowledge about plugin internal will not understand thanks :) -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

2 1

[Maria-developers] help with auth plugins
by Roberto Spadim 17 Oct '14

17 Oct '14

Hi guys! :) i need some help... i got the auth_dialog and i'm including some features to implement otp (one time password) my question is... what ".h" file should i include to allow read/write from any_table from mysql schema? i checked some read/write from selftunning optimizer ( https://github.com/igniting/server/blob/3d101751d27215ccf88c6c9701ca113de4a… ) but i didn't understand what should be included i'm getting messages like: In file included from /home/rspadim/mariadb/mariadb-10.1/server/sql/opt_range.h:36:0, from /home/rspadim/mariadb/mariadb-10.1/server/plugin/auth_otp/otp_auth.cc:26: --- my current source code (i don't know if .h order is ok): INCLUDES: #include "unireg.h" <- line 20 #include "sql_base.h" #include "lex.h" #include "sql_table.h" #include "sql_priv.h" #include "sql_class.h" #include "opt_range.h" <- line 26 #include "records.h" #include <mysql/plugin_auth.h> #include <string.h> #include <stdio.h> #include <stdlib.h> #include <mysql/auth_dialog_client.h> -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

[Maria-developers] Two bugs in MRR in need of review/comment
by Jeremy Cole 16 Oct '14

16 Oct '14

Hello, I filed these two bugs against multi-range read (MRR) in MariaDB 10.0.12 which affected us and could use some review of the patch for correctness, along with development of a proper test case: MDEV-6878 <https://mariadb.atlassian.net/browse/MDEV-6878>: Use of uninitialized saved_primary_key in Mrr_ordered_index_reader::resume_read() MDEV-6879 <https://mariadb.atlassian.net/browse/MDEV-6879>: Dereference of NULL primary_file->table in DsMrr_impl::get_disk_sweep_mrr_cost() Could someone please take a look at them? Regards, Jeremy

1 0

Re: [Maria-developers] MDEV-5359 CREATE OR REPLACE, CREATE IF NOT EXISTS, DROP IF EXISTS
by Sergei Golubchik 15 Oct '14

15 Oct '14

Hi, Alexander! This is all pretty good. Great work, thanks. I didn't have many comments. Few about small stylistic changes, and a couple of more serious ones - about tests and about events. See below. On Oct 02, Alexander Barkov wrote: > diff --git a/mysql-test/r/create_drop_db.result b/mysql-test/r/create_drop_db.result > new file mode 100644 > index 0000000..e7d4c18 > --- /dev/null > +++ b/mysql-test/r/create_drop_db.result > @@ -0,0 +1,17 @@ > +CREATE DATABASE db1; > +CREATE DATABASE IF NOT EXISTS db1; > +Warnings: > +Note 1007 Can't create database 'db1'; database exists > +CREATE OR REPLACE DATABASE db1; > +CREATE OR REPLACE DATABASE IF NOT EXISTS db2; > +ERROR HY000: Incorrect usage of OR REPLACE and IF NOT EXISTS > +DROP DATABASE db1; > +DROP DATABASE IF EXISTS db1; > +Warnings: > +Note 1008 Can't drop database 'db1'; database doesn't exist > +DROP DATABASE db1; > +ERROR HY000: Can't drop database 'db1'; database doesn't exist > +CREATE OR REPLACE DATABASE db1; > +CREATE DATABASE db1; > +ERROR HY000: Can't create database 'db1'; database exists > +DROP DATABASE IF EXISTS db1; I don't see where this test verifies that CREATE OR REPLACE actually replaces a database. Add the following: create database; create table; create or replace database; the table must not exist now; and add a test for create if not exists when a database does not exist. you can combine it with a previous test, if you'd like. > diff --git a/mysql-test/r/create_drop_event.result b/mysql-test/r/create_drop_event.result > new file mode 100644 > index 0000000..40d16e6 > --- /dev/null > +++ b/mysql-test/r/create_drop_event.result > @@ -0,0 +1,31 @@ > +SET GLOBAL event_scheduler=off; > +CREATE DATABASE db1; > +CREATE EVENT event_1 ON SCHEDULE EVERY 1 SECOND DO DROP DATABASE db1; > +SHOW DATABASES LIKE 'db1'; > +Database (db1) > +db1 > +SET GLOBAL event_scheduler=on; > +SHOW DATABASES LIKE 'db1'; > +Database (db1) > +SET GLOBAL event_scheduler=off; > +DROP DATABASE db1; > +ERROR HY000: Can't drop database 'db1'; database doesn't exist > +CREATE EVENT event_1 ON SCHEDULE EVERY 1 SECOND DO DROP DATABASE db1; > +ERROR HY000: Event 'event_1' already exists > +CREATE EVENT IF NOT EXISTS event_1 ON SCHEDULE EVERY 1 SECOND DO DROP DATABASE db1; > +Warnings: > +Note 1537 Event 'event_1' already exists > +CREATE OR REPLACE EVENT IF NOT EXISTS event_1 ON SCHEDULE EVERY 1 SECOND DO DROP DATABASE db1; > +ERROR HY000: Incorrect usage of OR REPLACE and IF NOT EXISTS > +CREATE OR REPLACE EVENT event_1 ON SCHEDULE EVERY 1 SECOND DO DROP DATABASE db1; again, same thing. I don't see a test that the command actually worked. please use different event definitions in all three create variants. and add show create event after every create to verify that create if not exists did *not* modify the event, but create or replace did modify it. > +CREATE DATABASE db1; > +SHOW DATABASES LIKE 'db1'; > +Database (db1) > +db1 > +SET GLOBAL event_scheduler=on; > +SHOW DATABASES LIKE 'db1'; > +Database (db1) > +SET GLOBAL event_scheduler=off; > +DROP DATABASE db1; > +ERROR HY000: Can't drop database 'db1'; database doesn't exist > +DROP EVENT IF EXISTS event_1; > diff --git a/mysql-test/r/create_drop_function.result b/mysql-test/r/create_drop_function.result > new file mode 100644 > index 0000000..750d179 > --- /dev/null > +++ b/mysql-test/r/create_drop_function.result > @@ -0,0 +1,34 @@ > +SET timestamp=UNIX_TIMESTAMP('2014-09-30 08:00:00'); > +CREATE FUNCTION f1(str char(20)) > +RETURNS CHAR(100) > +RETURN CONCAT('Hello, ', str, '!'); > +SELECT * FROM mysql.proc WHERE name like 'f1'; > +db name type specific_name language sql_data_access is_deterministic security_type param_list returns body definer created modified sql_mode comment character_set_client collation_connection db_collation body_utf8 > +test f1 FUNCTION f1 SQL CONTAINS_SQL NO DEFINER str char(20) char(100) CHARSET latin1 RETURN CONCAT('Hello, ', str, '!') root@localhost 2014-09-30 08:00:00 2014-09-30 08:00:00 latin1 latin1_swedish_ci latin1_swedish_ci RETURN CONCAT('Hello, ', str, '!') > +SELECT f1('world'); > +f1('world') > +Hello, world! > +CREATE FUNCTION f1(str char(20)) > +RETURNS TEXT > +RETURN CONCAT('Hello, ', str, '!'); > +ERROR 42000: FUNCTION f1 already exists > +CREATE FUNCTION IF NOT EXISTS f1(str char(20)) > +RETURNS CHAR(100) > +RETURN CONCAT('Hello, ', str, '!'); > +Warnings: > +Note 1304 FUNCTION f1 already exists same. please *different* definitions in different create variants. and some verification that the definition changed (or didn't change) - like show create or select f1(); > +CREATE OR REPLACE FUNCTION IF NOT EXISTS f1(str char(20)) > +RETURNS CHAR(100) > +RETURN CONCAT('Hello, ', str, '!'); > +ERROR HY000: Incorrect usage of OR REPLACE and IF NOT EXISTS > +CREATE OR REPLACE FUNCTION f1(str char(20)) > +RETURNS CHAR(100) > +RETURN CONCAT('Hello, ', str, '!'); > +DROP FUNCTION f1; > +CREATE FUNCTION IF NOT EXISTS f1(str char(20)) > +RETURNS CHAR(100) > +RETURN CONCAT('Hello, ', str, '!'); > +SELECT f1('world'); > +f1('world') > +Hello, world! > +DROP FUNCTION f1; > diff --git a/mysql-test/r/create_drop_index.result b/mysql-test/r/create_drop_index.result > new file mode 100644 > index 0000000..23f6388 > --- /dev/null > +++ b/mysql-test/r/create_drop_index.result > @@ -0,0 +1,34 @@ > +CREATE TABLE t1(id INT); > +CREATE INDEX idx ON t1(id); > +ALTER TABLE t1 ADD INDEX idx(id); > +ERROR 42000: Duplicate key name 'idx' > +ALTER TABLE t1 ADD INDEX IF NOT EXISTS idx(id); > +Warnings: > +Note 1061 Duplicate key name 'idx' > +CREATE OR REPLACE INDEX idx2 ON t1(id); same comment here (and everywhere below) > +Warnings: > +Note 1831 Duplicate index 'idx2' defined on the table 'test.t1'. This is deprecated and will be disallowed in a future release. > +ALTER TABLE t1 ADD INDEX IF NOT EXISTS idx2(id); > +Warnings: > +Note 1061 Duplicate key name 'idx2' > +CREATE OR REPLACE UNIQUE INDEX idx ON t1(id); > +CREATE INDEX idx ON t1(id); > +ERROR 42000: Duplicate key name 'idx' > +CREATE TABLE t2(col INT); > +CREATE OR REPLACE INDEX idx ON t2(col); > +CREATE INDEX idx ON t2(col); > +ERROR 42000: Duplicate key name 'idx' > +# Is this correct behaviour? > +ALTER TABLE t2 ADD FOREIGN KEY fk(col) REFERENCES t1(id); > +CREATE INDEX fk ON t2(col); > +Warnings: > +Note 1831 Duplicate index 'fk' defined on the table 'test.t2'. This is deprecated and will be disallowed in a future release. > +# Clearing up > +DROP INDEX idx ON t1; > +DROP TABLE t1; > +ALTER TABLE t1 DROP INDEX idx2; > +ERROR 42S02: Table 'test.t1' doesn't exist > +ALTER TABLE t2 DROP INDEX idx; > +ALTER TABLE t2 DROP INDEX fk; > +ALTER TABLE t2 DROP FOREIGN KEY fk; > +DROP TABLE t2; > diff --git a/mysql-test/r/proc_func.result b/mysql-test/r/proc_func.result > new file mode 100644 > index 0000000..1105ded > --- /dev/null > +++ b/mysql-test/r/proc_func.result > @@ -0,0 +1,63 @@ > +# > +# Testing CREATE PROCEDURE IF NOT EXISTS > +# > +DROP TABLE IF EXISTS t1; > +DROP PROCEDURE IF EXISTS proc1; > +CREATE TABLE t1 (id INT); > +CREATE PROCEDURE proc1 (OUT cnt INT) BEGIN SELECT COUNT(*) INTO cnt FROM t1; END$$ how is this test (and other tests below) different from create_drop_xxx tests above? > +CALL proc1(@cnt); > +SELECT @cnt; > +@cnt > +0 > +INSERT INTO t1 VALUES (1), (2), (3); > +CALL proc1(@cnt); > +SELECT @cnt; > +@cnt > +3 > +CREATE PROCEDURE proc1 (OUT cnt INT) BEGIN SELECT COUNT(*) INTO cnt FROM t1; END$$ > diff --git a/sql/event_db_repository.cc b/sql/event_db_repository.cc > index b20269b..01cdabb 100644 > --- a/sql/event_db_repository.cc > +++ b/sql/event_db_repository.cc > @@ -684,18 +684,28 @@ Event_db_repository::create_event(THD *thd, Event_parse_data *parse_data, > DBUG_PRINT("info", ("check existance of an event with the same name")); > if (!find_named_event(parse_data->dbname, parse_data->name, table)) > { > - if (create_if_not) > + if (thd->lex->is_create_or_replace()) > + { > + if ((ret= table->file->ha_delete_row(table->record[0]))) > + { > + table->file->print_error(ret, MYF(0)); > + goto end; > + } > + } > + else if (create_if_not) > { > *event_already_exists= true; > push_warning_printf(thd, Sql_condition::WARN_LEVEL_NOTE, > ER_EVENT_ALREADY_EXISTS, ER(ER_EVENT_ALREADY_EXISTS), > parse_data->name.str); > ret= 0; > + goto end; > } > else > + { > my_error(ER_EVENT_ALREADY_EXISTS, MYF(0), parse_data->name.str); > - > - goto end; > + goto end; > + } > } else > *event_already_exists= false; I wonder whether this works as it should. On create-or-replace you delete the old event row and a new one is supposedly inserted. So, the table is fine. But event_already_exists will stay unset, undefined, actually, because the caller didn't set it either. If you set it to TRUE, then the caller won't update in-memory even list, so your new event won't be enabled. If you set it to FALSE, it will try to insert a new event without deleting the old one and I don't know what will happen (either both will stay active or the insertion fails). Do you have a test to verify that after CREATE OR REPLACE EVENT the old event no longer exists and the new event does exist and is executed as expected? > diff --git a/sql/share/errmsg-utf8.txt b/sql/share/errmsg-utf8.txt > index ed7976e..1cfaaa4 100644 > --- a/sql/share/errmsg-utf8.txt > +++ b/sql/share/errmsg-utf8.txt > @@ -7110,3 +7110,11 @@ ER_IT_IS_A_VIEW 42S02 > eng "'%-.192s' is a view" > ER_SLAVE_SKIP_NOT_IN_GTID > eng "When using GTID, @@sql_slave_skip_counter can not be used. Instead, setting @@gtid_slave_pos explicitly can be used to skip to after a given GTID position." > +ER_USER_CREATE_EXISTS > + eng "Can't create user '%-.64s'@'%-.64s'; it already exists" > +ER_USER_DROP_EXISTS > + eng "Can't drop user '%-.64s'@'%-.64s'; it doesn't exist" > +ER_ROLE_CREATE_EXISTS > + eng "Can't create role '%-.64s'; it already exists" > +ER_ROLE_DROP_EXISTS > + eng "Can't drop role '%-.64s'; it doesn't exist" just a reminder - when merging (preferrably - rebasing) don't forget to put these errror messages after all 10.0 error messages. > diff --git a/sql/sp.cc b/sql/sp.cc > index 188b311..058a617 100644 > --- a/sql/sp.cc > +++ b/sql/sp.cc > @@ -835,6 +835,8 @@ db_load_routine(THD *thd, stored_procedure_type type, > > thd->lex= &newlex; > newlex.current_select= NULL; > + // Resetting all the flags in create_info > + newlex.create_info.options= 0; Isn't it safer to reset options in the parser, after CREATE clause? Like | CREATE { Lex->create_info.options= 0; } opt_or_replace ... Ok, see my very last comment in this review. > defstr.set_charset(creation_ctx->get_client_cs()); > > @@ -1015,6 +1063,47 @@ sp_create_routine(THD *thd, stored_procedure_type type, sp_head *sp) > ret= SP_OPEN_TABLE_FAILED; > else > { > + /* Checking if the routine already exists */ > + if (db_find_routine_aux(thd, type, lex->spname, table) == SP_OK) > + { > + if (lex->is_create_or_replace()) > + { > + if ((ret= sp_drop_routine_internal(thd, type, lex->spname, table))) > + goto done; > + } > + else if (lex->is_create_if_not_exists()) > + { > + push_warning_printf(thd, Sql_condition::WARN_LEVEL_NOTE, > + ER_SP_ALREADY_EXISTS, ER(ER_SP_ALREADY_EXISTS), > + type == TYPE_ENUM_FUNCTION ? > + "FUNCTION" : "PROCEDURE", > + lex->spname->m_name.str); > + > + ret= SP_OK; > + > + // Setting retstr as it is used for logging. > + if (sp->m_type == TYPE_ENUM_FUNCTION) > + { > + sp_returns_type(thd, retstr, sp); this is ok > + store_failed= store_failed || > + table->field[MYSQL_PROC_FIELD_RETURNS]-> > + store(retstr.ptr(), retstr.length(), system_charset_info); but why do you need to update table->field[MYSQL_PROC_FIELD_RETURNS] ? > + } > + > + if (store_failed) > + { > + ret= SP_FLD_STORE_FAILED; > + goto done; > + } > + goto log; > + } > + else > + { > + ret= SP_WRITE_ROW_FAILED; > + goto done; > + } > + } > + > restore_record(table, s->default_values); // Get default values for fields > > /* NOTE: all needed privilege checks have been already done. */ > diff --git a/sql/sql_acl.cc b/sql/sql_acl.cc > index 5f077c2..1e156be 100644 > --- a/sql/sql_acl.cc > +++ b/sql/sql_acl.cc > @@ -9148,6 +9148,42 @@ end: > DBUG_RETURN(result); > } > > + > +class Warning_user > +{ > +private: > + bool m_handle_as_role; > + void push_warning_role(THD *thd, uint err, const LEX_USER *user) > + { > + push_warning_printf(thd, Sql_condition::WARN_LEVEL_NOTE, > + err, ER(err), user->user.str); > + } > + void push_warning_user(THD *thd, uint err, const LEX_USER *user) > + { > + push_warning_printf(thd, Sql_condition::WARN_LEVEL_NOTE, > + err, ER(err), user->user.str, user->host.str); > + } > +public: > + Warning_user(bool handle_as_role) > + :m_handle_as_role(handle_as_role) > + { } > + void push_create_exists(THD *thd, const LEX_USER *user) > + { > + if (m_handle_as_role) > + push_warning_role(thd, ER_ROLE_CREATE_EXISTS, user); > + else > + push_warning_user(thd, ER_USER_CREATE_EXISTS, user); > + } > + void push_drop_doesnt_exist(THD *thd, const LEX_USER *user) > + { > + if (m_handle_as_role) > + push_warning_role(thd, ER_ROLE_DROP_EXISTS, user); > + else > + push_warning_user(thd, ER_USER_DROP_EXISTS, user); > + } > +}; > + > + > /* > Create a list of users. > > @@ -9215,10 +9252,26 @@ bool mysql_create_user(THD *thd, List <LEX_USER> &list, bool handle_as_role) > */ > if (handle_grant_data(tables, 0, user_name, NULL)) > { > - append_user(thd, &wrong_users, user_name); > - > - result= TRUE; > - continue; > + if (thd->lex->is_create_or_replace()) > + { > + if (handle_grant_data(tables, 1, user_name, NULL) <= 0) > + { > + append_user(thd, &wrong_users, user_name); I don't understand that at all. If user creation failed, you try to drop it, but don't create a user again, after the old one was dropped. How could it work? > + result= TRUE; > + continue; > + } > + } > + else if (thd->lex->is_create_if_not_exists()) > + { > + Warning_user(handle_as_role).push_create_exists(thd, user_name); Eh? Why do you need a class for that? Especially as it's temporarily instantiated here, and just to push a warning. Please, remove it. > + continue; > + } > + else > + { > + append_user(thd, &wrong_users, user_name); > + result= TRUE; > + continue; > + } > } > > some_users_created= TRUE; > @@ -9268,7 +9321,7 @@ bool mysql_create_user(THD *thd, List <LEX_USER> &list, bool handle_as_role) > (handle_as_role) ? "CREATE ROLE" : "CREATE USER", > wrong_users.c_ptr_safe()); > > - if (some_users_created) > + if (some_users_created || !result) What is that for? To binlog CREATE IF NOT EXISTS? > result |= write_bin_log(thd, FALSE, thd->query(), thd->query_length()); > > mysql_rwlock_unlock(&LOCK_grant); > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index 922f0d7..648fd48 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -3855,7 +3855,9 @@ end_with_restore_list: > } > } > #endif > - if (check_access(thd, CREATE_ACL, lex->name.str, NULL, NULL, 1, 0)) > + if (check_access(thd, lex->is_create_or_replace() ? > + CREATE_ACL | DROP_ACL : CREATE_ACL, good > + lex->name.str, NULL, NULL, 1, 0)) > break; > res= mysql_create_db(thd, lex->name.str, &create_info, 0); > break; > @@ -3995,9 +3997,8 @@ end_with_restore_list: > switch (lex->sql_command) { > case SQLCOM_CREATE_EVENT: > { > - bool if_not_exists= (lex->create_info.options & > - HA_LEX_CREATE_IF_NOT_EXISTS); > - res= Events::create_event(thd, lex->event_parse_data, if_not_exists); > + res= Events::create_event(thd, lex->event_parse_data, > + lex->is_create_if_not_exists()); This doesn't seem to be particularly logical. You pass lex->is_create_if_not_exists)() here an argument, but you check thd->lex->is_create_or_replace() in the Event_db_repository::create_event anyway. I'd suggest not to pass lex->is_create_if_not_exists() as an argument, and check both in Event_db_repository::create_event(). > break; > } > case SQLCOM_ALTER_EVENT: > @@ -6514,7 +6520,11 @@ bool add_field_to_list(THD *thd, LEX_STRING *field_name, enum_field_types type, > lex->col_list.push_back(new Key_part_spec(*field_name, 0)); > key= new Key(Key::PRIMARY, null_lex_str, > &default_key_create_info, > - 0, lex->col_list, NULL, lex->check_exists); > + 0, lex->col_list, NULL); > + > + if (lex->check_exists) > + key->key_create_info.options = HA_LEX_CREATE_IF_NOT_EXISTS; Hmm, is it for ALTER TABLE t1 ADD COLUMN IF NOT EXISTS foo INT PRIMARY KEY ? Do you have test cases for it? What does it do if the column exists but the key doesn't? What if the key exists and the column doens't? > + > lex->alter_info.key_list.push_back(key); > lex->col_list.empty(); > } > @@ -6524,7 +6534,11 @@ bool add_field_to_list(THD *thd, LEX_STRING *field_name, enum_field_types type, > lex->col_list.push_back(new Key_part_spec(*field_name, 0)); > key= new Key(Key::UNIQUE, null_lex_str, > &default_key_create_info, 0, > - lex->col_list, NULL, lex->check_exists); > + lex->col_list, NULL); > + > + if (lex->check_exists) > + key->key_create_info.options = HA_LEX_CREATE_IF_NOT_EXISTS; Same, needs tests (if there aren't). > + > lex->alter_info.key_list.push_back(key); > lex->col_list.empty(); > } > diff --git a/sql/sql_table.cc b/sql/sql_table.cc > index 8af77d9..b10c6d7 100644 > --- a/sql/sql_table.cc > +++ b/sql/sql_table.cc > @@ -5846,6 +5846,8 @@ handle_if_exists_options(THD *thd, TABLE *table, Alter_info *alter_info) > } > if (remove_key) > { take care when merging. "if (remove_key)" is not in the current 10.1, instead there's "remove_key:" label and goto. > + if (key->is_create_if_not_exists()) > + { > push_warning_printf(thd, Sql_condition::WARN_LEVEL_NOTE, > ER_DUP_KEYNAME, ER(ER_DUP_KEYNAME), keyname); > key_it.remove(); > diff --git a/sql/sql_trigger.cc b/sql/sql_trigger.cc > index 70ac626..b337451 100644 > --- a/sql/sql_trigger.cc > +++ b/sql/sql_trigger.cc > @@ -607,6 +607,89 @@ end: > DBUG_RETURN(result); > } > > +/** > + Build stmt_query to write it in the bin-log. > + > + @param thd current thread context (including trigger definition in > + LEX) > + @param tables table list containing one open table for which the > + trigger is created. > + @param[out] stmt_query after successful return, this string contains > + well-formed statement for creation this trigger. > + > + @note > + - Assumes that trigger name is fully qualified. > + - NULL-string means the following LEX_STRING instance: > + { str = 0; length = 0 }. > + - In other words, definer_user and definer_host should contain > + simultaneously NULL-strings (non-SUID/old trigger) or valid strings > + (SUID/new trigger). > + > + @retval > + False success > + @retval > + True error > +*/ > +static bool build_trig_stmt_query(THD *thd, TABLE_LIST *tables, > + String *stmt_query, > + LEX_STRING *trg_definer, > + char trg_definer_holder[]) > +{ > + LEX *lex = thd->lex; > + LEX_STRING definer_user; > + LEX_STRING definer_host; > + > + if (trg_definer == NULL && > + !(trg_definer= alloc_lex_string(&tables->table->mem_root))) > + return true; > + > + if (lex->sphead->m_chistics->suid != SP_IS_NOT_SUID) > + { > + /* SUID trigger. */ > + definer_user= lex->definer->user; > + definer_host= lex->definer->host; > + lex->definer->set_lex_string(trg_definer, trg_definer_holder); > + } > + else > + { > + /* non-SUID trigger. */ > + definer_user.str= 0; > + definer_user.length= 0; > + > + definer_host.str= 0; > + definer_host.length= 0; > + > + trg_definer->str= (char*) ""; > + trg_definer->length= 0; Why do you bother setting definer here? You can simply do below (see below): > + } > + > + /* > + Create well-formed trigger definition query. Original query is not > + appropriated, because definer-clause can be not truncated. > + */ > + > + stmt_query->append(STRING_WITH_LEN("CREATE ")); > + > + if (lex->is_create_or_replace()) > + stmt_query->append(STRING_WITH_LEN("OR REPLACE ")); > + > + if (lex->sphead->m_chistics->suid != SP_IS_NOT_SUID) > + { > + /* > + Append definer-clause if the trigger is SUID (a usual trigger in > + new MySQL versions). > + */ > + append_definer(thd, stmt_query, &definer_user, &definer_host); here: append_definer(thd, stmt_query, &lex->definer->user, &lex->definer->host); and set trg_definer: lex->definer->set_lex_string(trg_definer, trg_definer_holder); } else trg_definer= empty_lex_string; > + } > + > + LEX_STRING stmt_definition; > + stmt_definition.str= (char*) thd->lex->stmt_definition_begin; > + stmt_definition.length= thd->lex->stmt_definition_end - > + thd->lex->stmt_definition_begin; > + trim_whitespace(thd->charset(), & stmt_definition); > + stmt_query->append(stmt_definition.str, stmt_definition.length); > + return false; > +} > > /** > Create trigger for table. > @@ -639,16 +722,16 @@ bool Table_triggers_list::create_trigger(THD *thd, TABLE_LIST *tables, > char file_buff[FN_REFLEN], trigname_buff[FN_REFLEN]; > LEX_STRING file, trigname_file; > LEX_STRING *trg_def; > - LEX_STRING definer_user; > - LEX_STRING definer_host; > ulonglong *trg_sql_mode; > char trg_definer_holder[USER_HOST_BUFF_SIZE]; > - LEX_STRING *trg_definer; > + LEX_STRING *trg_definer= NULL; > Item_trigger_field *trg_field; > struct st_trigname trigname; > LEX_STRING *trg_client_cs_name; > LEX_STRING *trg_connection_cl_name; > LEX_STRING *trg_db_cl_name; > + sp_head *trg_head= bodies[lex->trg_chistics.event] > + [lex->trg_chistics.action_time]; trg_head as an element in a bodies array? May be trg_body instead? > > if (check_for_broken_triggers()) > return true; > diff --git a/sql/sql_view.cc b/sql/sql_view.cc > index a18193c..52b534e 100644 > --- a/sql/sql_view.cc > +++ b/sql/sql_view.cc > @@ -858,6 +863,14 @@ static int mysql_register_view(THD *thd, TABLE_LIST *view, > view->definer.host= lex->definer->host; > view->view_suid= lex->create_view_suid; > view->with_check= lex->create_view_check; > + > + DBUG_EXECUTE_IF("simulate_register_view_failure", > + my_error(ER_OUT_OF_RESOURCES, MYF(0));); > + DBUG_EXECUTE_IF("simulate_register_view_failure", > + error= -1;); > + DBUG_EXECUTE_IF("simulate_register_view_failure", > + goto err;); combine these in one DBUG_EXECUTE_IF: DBUG_EXECUTE_IF("simulate_register_view_failure", { my_error(ER_OUT_OF_RESOURCES, MYF(0)); error= -1; goto err; }); > + > if ((view->updatable_view= (can_be_merged && > view->algorithm != VIEW_ALGORITHM_TMPTABLE))) > { > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index d6b3fa4..7da047f 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -752,8 +752,18 @@ bool setup_select_in_parentheses(LEX *lex) > return FALSE; > } > > -static bool add_create_index_prepare (LEX *lex, Table_ident *table) > +static bool add_create_index_prepare (LEX *lex, Table_ident *table, > + uint create_or_replace, > + uint create_if_not_exists) > { > + if (create_or_replace && create_if_not_exists) > + { > + my_error(ER_WRONG_USAGE, MYF(0), "OR REPLACE", "IF NOT EXISTS"); > + return TRUE; > + } I'd rather see this check in the opt_if_not_exists rule, in only one place, that is. For this to work, though, the result of create_or_replace rule must be stored somewhere where opt_if_not_exists can access it. For example, in Lex->create_info.options. > + > + lex->key_create_info.options = (create_or_replace | create_if_not_exists); > + > lex->sql_command= SQLCOM_CREATE_INDEX; > if (!lex->current_select->add_table_to_list(lex->thd, table, NULL, > TL_OPTION_UPDATING, Regards, Sergei

1 0

[Maria-developers] MariaDB 10.0 on POWER
by Colin Charles 14 Oct '14

14 Oct '14

Hi, I'm sure you have read: https://www.flamingspork.com/blog/2014/10/08/mariadb-10-0-on-power/ For XL: There’s obviously still some work in progress going on, but my initial results show performance within around 10% of MySQL, so with a bit of work we will hopefully see MariaDB reach performance parity. For Serg: One interesting find was the code to account for thread memory usage uses a single atomic variable: this does not scale and does end up showing up on profiles. It isn't clear 10% loss to which MySQL, but comparing to 5.6 or 5.7, this has to "just work better" Thanks -- Colin Charles, Chief Evangelist, MariaDB Corporation blog: http://bytebot.net/blog/| t: +6-012-204-3201 | Skype: colincharles

2 1

Re: [Maria-developers] [Commits] 47fea67: MDEV-12 OpenGIS: create required tables: GeometryColumns, related views.
by Sergei Golubchik 14 Oct '14

14 Oct '14

Hi, Alexey! On Sep 02, Alexey Botchkov wrote: > revision-id: 47fea6798f0f37b8a42ad047d7daf6ec9bb52813 > parent(s): e44751b65f4760067d15f8a526e8f97f84810c29 > committer: Alexey Botchkov > branch nick: > timestamp: 2014-09-02 11:31:26 +0500 > message: > > MDEV-12 OpenGIS: create required tables: GeometryColumns, related views. > GEOMETRY_COLUMNS and SPATIAL_REF_SYS tables added to the INFORMATION_SCHEMA. > > diff --git a/mysql-test/r/information_schema.result > b/mysql-test/r/information_schema.result > index 6df5b89..93c9e90 100644 > --- a/mysql-test/r/information_schema.result > +++ b/mysql-test/r/information_schema.result > @@ -43,7 +43,8 @@ create view v1 (c) as > SELECT table_name FROM information_schema.TABLES > WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', > 'mysqltest') AND > table_name not like 'ndb_%' AND table_name not like 'innodb_%' AND > -table_name not like 'xtradb_%'; > +table_name not like 'xtradb_%' AND > +table_name not like 'geometry%' AND table_name not like 'spatial%'; Why did you filter these tables out? Normally, only plugin-created tables are filtered out, because these plugins might be disabled in the build. But we don't ever build without geometry, do we? I'd keep these tables in the result. > select * from v1; > c > ALL_PLUGINS > diff --git a/mysql-test/t/gis.test b/mysql-test/t/gis.test > index c387069..1a07a33 100644 > --- a/mysql-test/t/gis.test > +++ b/mysql-test/t/gis.test > @@ -1461,3 +1461,7 @@ insert into t1 values(Geomfromtext('POLYGON((1 1, > 2 2, 2 1, 1 1))')); > drop table t1; > > --echo End of 5.5 tests > + > +SHOW CREATE TABLE information_schema.geometry_columns; > +SHOW CREATE TABLE information_schema.spatial_ref_sys; Please, add tests for the content of these tables. Then ok to push (assuming these new tests will work fine). Regards, Sergei

1 0

[Maria-developers] Is it soon 'end of this year'?
by Peter Laursen 13 Oct '14

13 Oct '14

Around ½ year ago I discussed with some people here (Sergei G and/or Georg and/or Wlad) about support for Oracle-style plugins in the MariaDB C-API. The plugins include 1 Windows authentication plugin 2) Clear text plugin. At that time I was told that it could be considered 'by end end of this year'. 1) Windows authentication plugin. Not very important, but Sergei told that he had been playing wiht it and it could probably be done in ~1 day. 2) This is used by Oracle for setting up LDAP.authentication. I am very well aware that there is criticism of Oracle's implementation. But this does not change the fact hat people use it with servers from Oracle and expect clients to work with it. We get request(s) for it almost every week. So this is most important (for us). Any chance that those can soon be prioritized? (if you feel your self mis-quoted then I apologize).. -- Peter --Webyog

3 12

[Maria-developers] doubt about MDL
by Roberto Spadim 13 Oct '14

13 Oct '14

hi guys, i'm reading https://github.com/MariaDB/server/blob/10.1/sql/mdl.h i'm with a doubt... today we have the metadata lock plugin to check what mdl we have... my doubt is, how to expose how many threads are sharing the mdl key, and how many threads are waiting the mdl key, should it be possible or only one threrad could use one mdl? -- Roberto Spadim

2 3

Re: [Maria-developers] Page encryption exception handling.
by Sergei Golubchik 13 Oct '14

13 Oct '14

Hi, Clemens! On Oct 13, Clemens Dörrhöfer wrote: > Hi Sergei, > > I included an error, which was triggered by a missing key in the > keyfile. The server will see a wrong space ID, since it has not been > decrypted, which from my point of view is the reason for mariadb to > crash. > Basically we run into a problem when trying to decrypt a page using > fil_decrypt_page theoretically we can catch this exception much > sooner, since all we need to know is the key id we need for decryption > / encryption and throw an exception as soon as we know that we cannot > handle a key in best case before we trigger any aio. > > fil_decrypt_page() at fil0pageencryption.cc:391 0x555555feeb91 > os_file_read_func() at os0file.cc:3.156 0x555555e22648 > pfs_os_file_read_func() at os0file.ic:288 0x555555fdb4fa > fil_read_first_page() at fil0fil.cc:2.156 0x555555fe1eb0 > fil_validate_single_table_tablespace() at fil0fil.cc:4.303 0x555555fe5fd0 > fil_load_single_table_tablespace() at fil0fil.cc:4.443 0x555555fe64af > fil_load_single_table_tablespaces() at fil0fil.cc:4.806 0x555555fe6d1a > recv_init_crash_recovery() at log0recv.cc:3.046 0x555555e14447 > recv_recovery_from_checkpoint_start_func() at log0recv.cc:3.360 0x555555e14c69 > innobase_start_or_create_for_mysql() at srv0start.cc:2.471 0x555555eeea65 This still doesn't show where it crashes, only where you detect the missing key. But anyway, I agree, if you can detect a missing key sooner, you'd better do that, instead of returning invalid page to XtraDB, because it used to crash the server in case of severe errors like that. Regards, Sergei

1 0

[Maria-developers] Isn't thsi a bug?
by Peter Laursen 13 Oct '14

13 Oct '14

Refer: http://bugs.mysql.com/bug.php?id=74238 here I prefer the 'relaxed parsing' in Maria DB. But look at this: CREATE TABLE `d` ( `rec_id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT, `datetime` DATETIME(4) DEFAULT NULL, `timestamp` TIMESTAMP(4) NOT NULL DEFAULT CURRENT_TIMESTAMP(5) ON UPDATE CURRENT_TIMESTAMP(4), PRIMARY KEY (`rec_id`) ) ENGINE=INNODB; -- it actually creates the table SHOW CREATE TABLE d; /* returns CREATE TABLE `d` ( `rec_id` int(10) unsigned NOT NULL AUTO_INCREMENT, `datetime` datetime(4) DEFAULT NULL, `timestamp` timestamp(4) NOT NULL DEFAULT CURRENT_TIMESTAMP(4) ON UPDATE CURRENT_TIMESTAMP(4), PRIMARY KEY (`rec_id`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 */ I don't think DEFAULT CURRENT_TIMESTAMP(5) should be accepted for the datatype DATETIME(4). What say? -- Peter -- Webyog

2 2

[Maria-developers] Fwd: [JIRA] (MDEV-6566) Different INSERT behaviour on bad bytes with and without character set conversion
by Alexander Barkov 13 Oct '14

13 Oct '14

Hi Sergei, I suggest to fix this starting from 10.1. -------- Original Message -------- Subject: [JIRA] (MDEV-6566) Different INSERT behaviour on bad bytes with and without character set conversion Date: Thu, 9 Oct 2014 21:03:51 +0300 (EEST) From: Sergei Golubchik (JIRA) <jira(a)mariadb.atlassian.net> To: bar(a)mariadb.org [ https://mariadb.atlassian.net/browse/MDEV-6566?page=com.atlassian.jira.plug… ] Sergei Golubchik updated MDEV-6566: ----------------------------------- Fix Version/s: (was: 5.5.40) 5.5.41 > Different INSERT behaviour on bad bytes with and without character set conversion > --------------------------------------------------------------------------------- > > Key: MDEV-6566 > URL: https://mariadb.atlassian.net/browse/MDEV-6566 > Project: MariaDB Server > Issue Type: Bug > Affects Versions: 5.3.12, 5.5.39, 10.0.13 > Reporter: Alexander Barkov > Assignee: Alexander Barkov > Fix For: 10.0.15, 5.5.41 > > > If I run this SQL script in a utf8 client: > {noformat} > SET NAMES utf8; > DROP TABLE IF EXISTS t1; > CREATE TABLE t1 ( > a VARCHAR(10) CHARACTER SET ucs2, > b VARCHAR(10) CHARACTER SET utf8 > ); > INSERT INTO t1 SELECT 'a 😁 b', 'a 😁 b'; > SHOW WARNINGS; > SELECT * FROM t1; > {noformat} > It displays the following warnings: > {noformat} > +---------+------+----------------------------------------------------------------------+ > | Level | Code | Message | > +---------+------+----------------------------------------------------------------------+ > | Warning | 1300 | Invalid utf8 character string: '\xF0\x9F\x98\x81 b' | > | Warning | 1300 | Invalid utf8 character string: '\xF0\x9F\x98\x81 b' | > | Warning | 1366 | Incorrect string value: '\xF0\x9F\x98\x81 b' for column 'a' at row 1 | > | Warning | 1265 | Data truncated for column 'b' at row 1 | > +---------+------+----------------------------------------------------------------------+{noformat} > with the following results: > {noformat} > +----------+------+ > | a | b | > +----------+------+ > | a ???? b | a | > +----------+------+ > {noformat} > Notice, the character '😁' is "U+1F601 GRINNING FACE WITH SMILING EYES". It is outside of the BMP range supported by MariaDB character sets "utf8" and "ucs2". > The two columns worked differently: > - The ucs2 column correctly treated '😁' as an invalid sequence of four bytes, > converted it into four question marks, and appended the character 'b' after them. > - The utf8 column just stopped on this character and lost the trailing 'b'. > The warnings are also different for the two columns. > As "utf8" and "ucs2" are equitable character sets that support exactly the > same character range and repertoire (just with a different encoding), > it would be fair to expect the same behaviour of the two columns. > The column "a" exposes a better behaviour: it preserves as much > information as possible. > The column "b" should be fixed to work in the same way with "a". > The reason of the failure is that the conversion code that is activated > for the column "a" (with conversion) uses an mb_wc()..mb_wc() loop, while the > code activated for the column "b" (without conversion) copies only a well formed > prefix. > The code without conversion should not stop on a bad byte sequence > and try to copy as much data as possible, like the conversion code does. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003)

2 1

[Maria-developers] Page encryption exception handling.
by Clemens Dörrhöfer 13 Oct '14

13 Oct '14

Hi community, I am working on implementing page encryption for XtraDB. Encryption itself works fine, now I am focusing on exception handling. My problem is that whenever page encryption is misconfigured it will crash the entire server, which is no acceptable behaviour. Example I encrypted a table with key id 42, I delete the key from the keypool thus I can no longer decrypt the table, because the appropriate key is missing. Ideally whenever a client tries to query data from an encrypted table, which cannot be decrypted, it will receive an error message informing the client that Table X could not be decrypted because key 42 is missing. Can you point me out to how to accomplish such a behaviour. Greetings Clemens Dörrhöfer

2 1

Re: [Maria-developers] [Commits] [svoj@mariadb.org: Rev 4437: Fixed mroonga build failure on Power8: define generic gcc version of in lp:maria/10.0]
by Kouhei Sutou 09 Oct '14

09 Oct '14

Hi Sergey, Thanks for the fix! I've merged your change into Groonga: https://github.com/groonga/groonga/commit/b3d38c5b9863cb7a9e93b9461c8b93fbf… Thanks, -- kou In <20141009083146.GA3231@june> "[Commits] [svoj(a)mariadb.org: Rev 4437: Fixed mroonga build failure on Power8: define generic gcc version of in lp:maria/10.0]" on Thu, 9 Oct 2014 12:31:47 +0400, Sergey Vojtovich <svoj(a)mariadb.org> wrote: > Hi Kentoku, > > FYI: I just pushed this fix to 10.0. There're also some test failures on Power8 > in mroonga/storage suite, mroonga/wrapper was 100% successful. > > Regards, > Sergey > > ----- Forwarded message from Sergey Vojtovich <svoj(a)mariadb.org> ----- > > Date: Thu, 9 Oct 2014 08:25:58 +0000 (UTC) > From: Sergey Vojtovich <svoj(a)mariadb.org> > To: commits(a)mariadb.org > Subject: [Commits] Rev 4437: Fixed mroonga build failure on Power8: define generic gcc version of in lp:maria/10.0 > > At lp:maria/10.0 > > ------------------------------------------------------------ > revno: 4437 > revision-id: svoj(a)mariadb.org-20141009082545-nt183b7y1gjihfjp > parent: psergey(a)askmonty.org-20141006112922-45pcytb993xcwh89 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0-mroonga > timestamp: Thu 2014-10-09 12:25:45 +0400 > message: > Fixed mroonga build failure on Power8: define generic gcc version of > GRN_SET_64BIT. > === modified file 'storage/mroonga/vendor/groonga/lib/groonga_in.h' > --- a/storage/mroonga/vendor/groonga/lib/groonga_in.h 2014-09-20 15:33:45 +0000 > +++ b/storage/mroonga/vendor/groonga/lib/groonga_in.h 2014-10-09 08:25:45 +0000 > @@ -505,6 +505,9 @@ typedef int grn_cond; > /* todo */ > # define GRN_SET_64BIT(p,v) \ > (void)atomic_swap_64(p, v) > +# elif defined(__ATOMIC_SEQ_CST) /* GCC atomic builtins */ > +# define GRN_SET_64BIT(p,v) \ > + __atomic_store_n(p, v, __ATOMIC_SEQ_CST) > # endif /* ATOMIC 64BIT SET */ > > # ifdef HAVE_MKOSTEMP > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits > > ----- End forwarded message ----- > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

2 2

[Maria-developers] INET6, data type plugins
by Alexander Barkov 09 Oct '14

09 Oct '14

Hello all, I have mostly implemented the server side part of the fixed length 16-byte INET6 data type (only some clean-ups left). MDEV-274 The data type for IPv6/IPv4 addresses in MariaDB https://mariadb.atlassian.net/browse/MDEV-274 I made it in a very new style, which is ready to data type plugins, as described in: MDEV-4912 Add a plugin to field types (column types) https://mariadb.atlassian.net/browse/MDEV-4912 It is done by introducing a new abstract super-class Type_handler with a number of virtual public methods (around 50 public methods at this point). To implement a new data type one just needs to implement an instantiable class derived from Type_handler and register the new data type in a special global registry. Everything start to work automatically, including (but not limited to): - correct comparison - hybrid type functions like COALESCE, LEAST - aggregate functions MAX() and MIN() - CAST to the new type - ORDER BY, both for field and for items - conversion to the standard data types - indexing and index search The definition of the instantiable class that actually implements INET6 looks like this: class Type_handler_inet6: public Type_handler { private: class Field_inet6: public Field { ... }; class Arg_comparator_inet6: public Arg_comparator { ... }; class Item_typecast_inet6: public Item_func { ... }; public: // various access methods }; It looks very nice for me so far :) I have some questions about the remaining things: 1. In my current patch the new data type INET6 is not loaded dynamically from a shared library. It's implemented inside mysqld. Is it Okey? Or would we prefer to make it as a plugin from the beginning? (in this case there's some more work left: the dynamic loader code, client-server protocol, PS protocol). 2. In case of pluggable data types we'll need some changes in the client-server protocol, to transfer data type names (and probably some more metadata) to the client side in some textual form. Any ideas on that? 3. PS protocol obviously needs two ways to bind data: in text format and in binary format. For example, in case of INET6 it would be nice to be able to bind parameters and fetch values using: - text representation "ffff::ffff" - and binary representation 0xFFFF000000000000000000000000FFFF. I can see some ways to do this: a. Every pluggable data type uses its own type code in MYSQL_BIND: char ip6[16]= "\xff\xff...\xff"; MYSQL_BIND bind; bind.buffer_type= 128; // This is INET6 bind.buffer= ip6; bind.length= sizeof(ip6); but the code (128) may vary between servers, depending on how the plugin is installed. b. Or we could introduce a singe new type code MYSQL_TYPE_RAW: char ip6[16]= "\xff\xff...\xff"; MYSQL_BIND bind; bind.buffer_type= MYSQL_TYPE_RAW; bind.buffer= ip6; bind.length= sizeof(ip6); The server should be able to treat a single type code with different data types without any problems, because it knows the parameter context. But this will not allow to mix values of different types though, for example insert an INET4 value into an INET6 column (or vice verse) won't be possible. c. Or we could support both "native" type codes and MYSQL_TYPE_RAW. 4. Do we need the INET4 data type (fixed length, 4 bytes)? 5. Do we need a PostgreSQL-alike INET data type which is able to store both INET4 and INET6 values (I guess this one should be variable length: 4-bytes for INET4 and 16 bytes for INET6 values. Any feedback would be appreciated. The current patch (attached) will be polished. but the general idea should already be clear from the current version. Thanks.

3 4

Re: [Maria-developers] [Commits] Rev 4391: MDEV-6676: Speculative parallel replication. in http://bazaar.launchpad.net/~maria-captains/maria/10.0
by Jonas Oreland 08 Oct '14

08 Oct '14

how about also adding a mode = 'auto' which currently can be == transaction, but in future could be extended to based on statistics flip actual modes back and forth ? /Jonas On Tue, Oct 7, 2014 at 4:20 PM, <knielsen(a)knielsen-hq.org> wrote: > At http://bazaar.launchpad.net/~maria-captains/maria/10.0 > > ------------------------------------------------------------ > revno: 4391 > revision-id: knielsen(a)knielsen-hq.org-20141007142037-y85xq099ltdiqno6 > parent: knielsen(a)knielsen-hq.org-20141003094323-nosx0y7vv0kjv20e > committer: Kristian Nielsen <knielsen(a)knielsen-hq.org> > branch nick: work-10.0-mdev6676 > timestamp: Tue 2014-10-07 16:20:37 +0200 > message: > MDEV-6676: Speculative parallel replication. > > Better syntax for configuring the parallel mode. Now it is a CHANGE > MASTER > option instead of a system variable, which makes it possible to configure > differently for different multi-source replication masters. > > Now also the domain-based replication mode can be disabled. > === modified file 'mysql-test/include/check-testcase.test' > --- a/mysql-test/include/check-testcase.test 2013-12-16 12:02:21 +0000 > +++ b/mysql-test/include/check-testcase.test 2014-10-07 14:20:37 +0000 > @@ -64,6 +64,7 @@ if ($tmp) > --echo Master_SSL_Crlpath # > --echo Using_Gtid No > --echo Gtid_IO_Pos # > + --echo Parallel_Mode domain,groupcommit > } > if (!$tmp) { > # Note: after WL#5177, fields 13-18 shall not be filtered-out. > > === modified file 'mysql-test/r/mysqld--help.result' > --- a/mysql-test/r/mysqld--help.result 2014-09-19 13:25:37 +0000 > +++ b/mysql-test/r/mysqld--help.result 2014-10-07 14:20:37 +0000 > @@ -915,17 +915,6 @@ > parallel replication thread when reading ahead in the > relay log looking for opportunities for parallel > replication. Only used when --slave-parallel-threads > 0. > - --slave-parallel-mode=name > - Controls what transactions are applied in parallel when > - using --slave-parallel-threads. Syntax: > - slave_paralle_mode=value[,value...], where "value" could > - be one or more of: "domain", to apply different > - replication domains in parallel; "groupcommit", to apply > - in parallel transactions that group-committed together on > - the master; "transactional", to optimistically try to > - apply all transactional DML in parallel; and "waiting" to > - extend "transactional" to even transactions that had to > - wait on the master. > --slave-parallel-threads=# > If non-zero, number of threads to spawn to apply in > parallel events on the slave that were group-committed on > @@ -1320,7 +1309,6 @@ slave-exec-mode STRICT > slave-max-allowed-packet 1073741824 > slave-net-timeout 3600 > slave-parallel-max-queued 131072 > -slave-parallel-mode domain,groupcommit > slave-parallel-threads 0 > slave-skip-errors (No default value) > slave-sql-verify-checksum TRUE > > === modified file 'mysql-test/suite/rpl/r/rpl_parallel.result' > --- a/mysql-test/suite/rpl/r/rpl_parallel.result 2014-08-15 > 09:31:13 +0000 > +++ b/mysql-test/suite/rpl/r/rpl_parallel.result 2014-10-07 > 14:20:37 +0000 > @@ -1,5 +1,5 @@ > -include/rpl_init.inc [topology=1->2] > -SET @old_parallel_threads=@@GLOBAL.slave_parallel_threads; > +include/master-slave.inc > +[connection master] > SET GLOBAL slave_parallel_threads=10; > ERROR HY000: This operation cannot be performed as you have a running > slave ''; run STOP SLAVE '' first > include/stop_slave.inc > @@ -923,8 +923,79 @@ SELECT * FROM t2 WHERE a >= 30 ORDER BY > 32 > 33 > 34 > +*** MDEV-6676 - test syntax of CHANGE MASTER TO PARALLEL_MODE=xxx *** > +Parallel_Mode = 'domain,groupcommit' > include/stop_slave.inc > -SET GLOBAL slave_parallel_threads=@old_parallel_threads; > +CHANGE MASTER TO parallel_mode=(,domain,groupcommit); > +ERROR 42000: You have an error in your SQL syntax; check the manual that > corresponds to your MariaDB server version for the right syntax to use near > 'domain,groupcommit)' at line 1 > +CHANGE MASTER TO parallel_mode=(domain,groupcommit,transactional); > +ERROR HY000: Invalid use of 'transactional' option for PARALLEL_MODE in > CHANGE MASTER TO > +CHANGE MASTER TO parallel_mode=(waiting,domain,transactional,waiting); > +ERROR HY000: Invalid use of 'waiting' option for PARALLEL_MODE in CHANGE > MASTER TO > +CHANGE MASTER TO parallel_mode=(domain,domain); > +ERROR HY000: Invalid use of 'domain' option for PARALLEL_MODE in CHANGE > MASTER TO > +CHANGE MASTER TO parallel_mode=(waiting,transactional,domain); > +Parallel_Mode = 'domain,transactional,waiting' > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > +Parallel_Mode = 'domain,groupcommit' > +*** MDEV-6676 - test that empty parallel_mode does not replicate in > parallel *** > +INSERT INTO t2 VALUES (40); > +include/save_master_gtid.inc > +CHANGE MASTER TO parallel_mode=(); > +SET @old_dbug= @@GLOBAL.debug_dbug; > +SET GLOBAL debug_dbug="+d,slave_crash_if_parallel_apply"; > +include/start_slave.inc > +include/sync_with_master_gtid.inc > +SELECT * FROM t2 WHERE a >= 40 ORDER BY a; > +a > +40 > +include/stop_slave.inc > +SET GLOBAL debug_dbug=@old_dbug; > +*** MDEV-6676 - test disabling domain-based parallel replication *** > +SET gtid_domain_id = 1; > +INSERT INTO t2 VALUES (41); > +INSERT INTO t2 VALUES (42); > +INSERT INTO t2 VALUES (43); > +INSERT INTO t2 VALUES (44); > +INSERT INTO t2 VALUES (45); > +INSERT INTO t2 VALUES (46); > +DELETE FROM t2 WHERE a >= 41; > +SET gtid_domain_id = 2; > +INSERT INTO t2 VALUES (41); > +INSERT INTO t2 VALUES (42); > +INSERT INTO t2 VALUES (43); > +INSERT INTO t2 VALUES (44); > +INSERT INTO t2 VALUES (45); > +INSERT INTO t2 VALUES (46); > +SET gtid_domain_id = 0; > +include/save_master_gtid.inc > +CHANGE MASTER TO parallel_mode=(groupcommit); > +include/start_slave.inc > +include/sync_with_master_gtid.inc > +SELECT * FROM t2 WHERE a >= 40 ORDER BY a; > +a > +40 > +41 > +42 > +43 > +44 > +45 > +46 > +include/stop_slave.inc > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > +include/start_slave.inc > +*** MDEV-6676 - test that parallel mode is saved correctly in master.info > across server restart *** > +Parallel_Mode = 'domain,groupcommit' > +include/stop_slave.inc > +CHANGE MASTER TO parallel_mode=(transactional,waiting); > +Parallel_Mode = 'transactional,waiting' > +include/start_slave.inc > +include/rpl_restart_server.inc [server_number=2] > +Parallel_Mode = 'transactional,waiting' > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > +include/start_slave.inc > +include/stop_slave.inc > +SET GLOBAL slave_parallel_threads=0; > include/start_slave.inc > SET DEBUG_SYNC= 'RESET'; > DROP function foo; > > === modified file 'mysql-test/suite/rpl/r/rpl_parallel_optimistic.result' > --- a/mysql-test/suite/rpl/r/rpl_parallel_optimistic.result 2014-09-24 > 19:25:10 +0000 > +++ b/mysql-test/suite/rpl/r/rpl_parallel_optimistic.result 2014-10-07 > 14:20:37 +0000 > @@ -2,11 +2,9 @@ include/rpl_init.inc [topology=1->2] > ALTER TABLE mysql.gtid_slave_pos ENGINE=InnoDB; > CREATE TABLE t1 (a int PRIMARY KEY, b INT) ENGINE=InnoDB; > SET @old_parallel_threads=@@GLOBAL.slave_parallel_threads; > -SET @old_mode=@@GLOBAL.slave_parallel_mode; > include/stop_slave.inc > SET GLOBAL slave_parallel_threads=10; > -SET GLOBAL slave_parallel_mode="domain,transactional"; > -CHANGE MASTER TO master_use_gtid=slave_pos; > +CHANGE MASTER TO master_use_gtid=slave_pos, > parallel_mode=(domain,transactional); > INSERT INTO t1 VALUES(1,1); > BEGIN; > INSERT INTO t1 VALUES(2,1); > @@ -288,8 +286,8 @@ SET GLOBAL binlog_format= @old_format; > SET GLOBAL tx_isolation= @old_isolation; > include/start_slave.inc > include/stop_slave.inc > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > SET GLOBAL slave_parallel_threads=@old_parallel_threads; > -SET GLOBAL slave_parallel_mode=@old_mode; > include/start_slave.inc > DROP TABLE t1, t2, t3; > include/rpl_end.inc > > === modified file > 'mysql-test/suite/rpl/r/rpl_parallel_optimistic_nobinlog.result' > --- a/mysql-test/suite/rpl/r/rpl_parallel_optimistic_nobinlog.result > 2014-09-24 19:25:10 +0000 > +++ b/mysql-test/suite/rpl/r/rpl_parallel_optimistic_nobinlog.result > 2014-10-07 14:20:37 +0000 > @@ -6,12 +6,10 @@ INSERT INTO t1 VALUES (1,0), (2,0), (3,0 > INSERT INTO t2 VALUES (1,0), (2,0); > SET @old_isolation= @@GLOBAL.tx_isolation; > SET @old_parallel_threads=@@GLOBAL.slave_parallel_threads; > -SET @old_mode=@@GLOBAL.slave_parallel_mode; > include/stop_slave.inc > SET GLOBAL TRANSACTION ISOLATION LEVEL READ COMMITTED; > SET GLOBAL slave_parallel_threads=10; > -SET GLOBAL slave_parallel_mode="domain,transactional,waiting"; > -CHANGE MASTER TO master_use_gtid=slave_pos; > +CHANGE MASTER TO master_use_gtid=slave_pos, > parallel_mode=(domain,transactional,waiting); > *** Test that we replicate correctly when using READ COMMITTED and > --log-slave-updates=0 on the slave *** > INSERT INTO t1 SELECT 4, COUNT(*) FROM t2; > INSERT INTO t2 SELECT 4, COUNT(*) FROM t1; > @@ -78,8 +76,8 @@ a b > 10 10 > include/stop_slave.inc > SET GLOBAL tx_isolation= @old_isolation; > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > SET GLOBAL slave_parallel_threads=@old_parallel_threads; > -SET GLOBAL slave_parallel_mode=@old_mode; > include/start_slave.inc > DROP TABLE t1, t2; > include/rpl_end.inc > > === modified file 'mysql-test/suite/rpl/t/rpl_parallel.test' > --- a/mysql-test/suite/rpl/t/rpl_parallel.test 2014-08-15 09:31:13 +0000 > +++ b/mysql-test/suite/rpl/t/rpl_parallel.test 2014-10-07 14:20:37 +0000 > @@ -1,13 +1,12 @@ > --source include/have_innodb.inc > --source include/have_debug.inc > --source include/have_debug_sync.inc > ---let $rpl_topology=1->2 > ---source include/rpl_init.inc > +--source include/master-slave.inc > > # Test various aspects of parallel replication. > > --connection server_2 > -SET @old_parallel_threads=@@GLOBAL.slave_parallel_threads; > +--let $old_parallel_threads=`SELECT @@GLOBAL.slave_parallel_threads` > --error ER_SLAVE_MUST_STOP > SET GLOBAL slave_parallel_threads=10; > --source include/stop_slave.inc > @@ -1466,9 +1465,100 @@ SET sql_slave_skip_counter= 1; > SELECT * FROM t2 WHERE a >= 30 ORDER BY a; > > > +--echo *** MDEV-6676 - test syntax of CHANGE MASTER TO PARALLEL_MODE=xxx > *** > +--connection server_2 > + > +--let $status_items= Parallel_Mode > +--source include/show_slave_status.inc > +--source include/stop_slave.inc > +--error ER_PARSE_ERROR > +CHANGE MASTER TO parallel_mode=(,domain,groupcommit); > +--error ER_INVALID_SLAVE_PARALLEL_MODE > +CHANGE MASTER TO parallel_mode=(domain,groupcommit,transactional); > +--error ER_INVALID_SLAVE_PARALLEL_MODE > +CHANGE MASTER TO parallel_mode=(waiting,domain,transactional,waiting); > +--error ER_INVALID_SLAVE_PARALLEL_MODE > +CHANGE MASTER TO parallel_mode=(domain,domain); > +CHANGE MASTER TO parallel_mode=(waiting,transactional,domain); > +--let $status_items= Parallel_Mode > +--source include/show_slave_status.inc > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > +--let $status_items= Parallel_Mode > +--source include/show_slave_status.inc > + > + > +--echo *** MDEV-6676 - test that empty parallel_mode does not replicate > in parallel *** > +--connection server_1 > +INSERT INTO t2 VALUES (40); > +--source include/save_master_gtid.inc > + > +--connection server_2 > +CHANGE MASTER TO parallel_mode=(); > +# Test that we do not use parallel apply, by injecting an unconditional > +# crash in the parallel apply code. > +SET @old_dbug= @@GLOBAL.debug_dbug; > +SET GLOBAL debug_dbug="+d,slave_crash_if_parallel_apply"; > +--source include/start_slave.inc > +--source include/sync_with_master_gtid.inc > +SELECT * FROM t2 WHERE a >= 40 ORDER BY a; > +--source include/stop_slave.inc > +SET GLOBAL debug_dbug=@old_dbug; > + > + > +--echo *** MDEV-6676 - test disabling domain-based parallel replication > *** > +--connection server_1 > +# Let's do a bunch of transactions that will conflict if run out-of-order > in > +# domain-based parallel replication mode. > +SET gtid_domain_id = 1; > +INSERT INTO t2 VALUES (41); > +INSERT INTO t2 VALUES (42); > +INSERT INTO t2 VALUES (43); > +INSERT INTO t2 VALUES (44); > +INSERT INTO t2 VALUES (45); > +INSERT INTO t2 VALUES (46); > +DELETE FROM t2 WHERE a >= 41; > +SET gtid_domain_id = 2; > +INSERT INTO t2 VALUES (41); > +INSERT INTO t2 VALUES (42); > +INSERT INTO t2 VALUES (43); > +INSERT INTO t2 VALUES (44); > +INSERT INTO t2 VALUES (45); > +INSERT INTO t2 VALUES (46); > +SET gtid_domain_id = 0; > +--source include/save_master_gtid.inc > +--connection server_2 > +CHANGE MASTER TO parallel_mode=(groupcommit); > +--source include/start_slave.inc > +--source include/sync_with_master_gtid.inc > +SELECT * FROM t2 WHERE a >= 40 ORDER BY a; > +--source include/stop_slave.inc > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > +--source include/start_slave.inc > + > + > +--echo *** MDEV-6676 - test that parallel mode is saved correctly in > master.info across server restart *** > +--connection server_2 > +--let $status_items= Parallel_Mode > +--source include/show_slave_status.inc > +--source include/stop_slave.inc > +CHANGE MASTER TO parallel_mode=(transactional,waiting); > +--let $status_items= Parallel_Mode > +--source include/show_slave_status.inc > +--source include/start_slave.inc > + > +--let $rpl_server_number= 2 > +--source include/rpl_restart_server.inc > + > +--connection server_2 > +--let $status_items= Parallel_Mode > +--source include/show_slave_status.inc > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > +--source include/start_slave.inc > + > + > --connection server_2 > --source include/stop_slave.inc > -SET GLOBAL slave_parallel_threads=@old_parallel_threads; > +eval SET GLOBAL slave_parallel_threads=$old_parallel_threads; > --source include/start_slave.inc > SET DEBUG_SYNC= 'RESET'; > > > === modified file 'mysql-test/suite/rpl/t/rpl_parallel_optimistic.test' > --- a/mysql-test/suite/rpl/t/rpl_parallel_optimistic.test 2014-09-24 > 19:25:10 +0000 > +++ b/mysql-test/suite/rpl/t/rpl_parallel_optimistic.test 2014-10-07 > 14:20:37 +0000 > @@ -11,11 +11,9 @@ CREATE TABLE t1 (a int PRIMARY KEY, b IN > --connection server_2 > --sync_with_master > SET @old_parallel_threads=@@GLOBAL.slave_parallel_threads; > -SET @old_mode=@@GLOBAL.slave_parallel_mode; > --source include/stop_slave.inc > SET GLOBAL slave_parallel_threads=10; > -SET GLOBAL slave_parallel_mode="domain,transactional"; > -CHANGE MASTER TO master_use_gtid=slave_pos; > +CHANGE MASTER TO master_use_gtid=slave_pos, > parallel_mode=(domain,transactional); > > > --connection server_1 > @@ -309,8 +307,8 @@ SET GLOBAL tx_isolation= @old_isolation; > > --connection server_2 > --source include/stop_slave.inc > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > SET GLOBAL slave_parallel_threads=@old_parallel_threads; > -SET GLOBAL slave_parallel_mode=@old_mode; > --source include/start_slave.inc > > --connection server_1 > > === modified file > 'mysql-test/suite/rpl/t/rpl_parallel_optimistic_nobinlog.test' > --- a/mysql-test/suite/rpl/t/rpl_parallel_optimistic_nobinlog.test > 2014-09-24 19:25:10 +0000 > +++ b/mysql-test/suite/rpl/t/rpl_parallel_optimistic_nobinlog.test > 2014-10-07 14:20:37 +0000 > @@ -16,12 +16,10 @@ INSERT INTO t2 VALUES (1,0), (2,0); > --sync_with_master > SET @old_isolation= @@GLOBAL.tx_isolation; > SET @old_parallel_threads=@@GLOBAL.slave_parallel_threads; > -SET @old_mode=@@GLOBAL.slave_parallel_mode; > --source include/stop_slave.inc > SET GLOBAL TRANSACTION ISOLATION LEVEL READ COMMITTED; > SET GLOBAL slave_parallel_threads=10; > -SET GLOBAL slave_parallel_mode="domain,transactional,waiting"; > -CHANGE MASTER TO master_use_gtid=slave_pos; > +CHANGE MASTER TO master_use_gtid=slave_pos, > parallel_mode=(domain,transactional,waiting); > > > --echo *** Test that we replicate correctly when using READ COMMITTED and > --log-slave-updates=0 on the slave *** > @@ -63,8 +61,8 @@ SELECT * FROM t2 ORDER BY a; > --connection server_2 > --source include/stop_slave.inc > SET GLOBAL tx_isolation= @old_isolation; > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > SET GLOBAL slave_parallel_threads=@old_parallel_threads; > -SET GLOBAL slave_parallel_mode=@old_mode; > --source include/start_slave.inc > > --connection server_1 > > === modified file 'sql/lex.h' > --- a/sql/lex.h 2014-02-02 09:00:36 +0000 > +++ b/sql/lex.h 2014-10-07 14:20:37 +0000 > @@ -194,6 +194,7 @@ static SYMBOL symbols[] = { > { "DISTINCTROW", SYM(DISTINCT)}, /* Access likes this */ > { "DIV", SYM(DIV_SYM)}, > { "DO", SYM(DO_SYM)}, > + { "DOMAIN", SYM(DOMAIN_SYM)}, > { "DOUBLE", SYM(DOUBLE_SYM)}, > { "DROP", SYM(DROP)}, > { "DUAL", SYM(DUAL_SYM)}, > @@ -257,6 +258,7 @@ static SYMBOL symbols[] = { > { "GRANT", SYM(GRANT)}, > { "GRANTS", SYM(GRANTS)}, > { "GROUP", SYM(GROUP_SYM)}, > + { "GROUPCOMMIT", SYM(GROUPCOMMIT_SYM)}, > { "HANDLER", SYM(HANDLER_SYM)}, > { "HARD", SYM(HARD_SYM)}, > { "HASH", SYM(HASH_SYM)}, > @@ -429,6 +431,7 @@ static SYMBOL symbols[] = { > { "PACK_KEYS", SYM(PACK_KEYS_SYM)}, > { "PAGE", SYM(PAGE_SYM)}, > { "PAGE_CHECKSUM", SYM(PAGE_CHECKSUM_SYM)}, > + { "PARALLEL_MODE", SYM(PARALLEL_MODE_SYM)}, > { "PARSER", SYM(PARSER_SYM)}, > { "PARSE_VCOL_EXPR", SYM(PARSE_VCOL_EXPR_SYM)}, > { "PARTIAL", SYM(PARTIAL)}, > @@ -650,6 +653,7 @@ static SYMBOL symbols[] = { > { "VIEW", SYM(VIEW_SYM)}, > { "VIRTUAL", SYM(VIRTUAL_SYM)}, > { "WAIT", SYM(WAIT_SYM)}, > + { "WAITING", SYM(WAITING_SYM)}, > { "WARNINGS", SYM(WARNINGS)}, > { "WEEK", SYM(WEEK_SYM)}, > { "WEIGHT_STRING", SYM(WEIGHT_STRING_SYM)}, > > === modified file 'sql/log_event.cc' > --- a/sql/log_event.cc 2014-09-25 13:47:58 +0000 > +++ b/sql/log_event.cc 2014-10-07 14:20:37 +0000 > @@ -12728,7 +12728,7 @@ bool rpl_get_position_info(const char ** > return FALSE; > #else > const Relay_log_info *rli= &(active_mi->rli); > - if (opt_slave_parallel_threads == 0) > + if (!rli->mi->using_parallel()) > { > *log_file_name= rli->group_master_log_name; > *log_pos= rli->group_master_log_pos + > > === modified file 'sql/mysqld.cc' > --- a/sql/mysqld.cc 2014-09-26 10:37:19 +0000 > +++ b/sql/mysqld.cc 2014-10-07 14:20:37 +0000 > @@ -562,8 +562,6 @@ ulong stored_program_cache_size= 0; > > ulong opt_slave_parallel_threads= 0; > ulong opt_slave_domain_parallel_threads= 0; > -ulonglong opt_slave_parallel_mode= > - SLAVE_PARALLEL_DOMAIN | SLAVE_PARALLEL_GROUPCOMMIT; > ulong opt_binlog_commit_wait_count= 0; > ulong opt_binlog_commit_wait_usec= 0; > ulong opt_slave_parallel_max_queued= 131072; > > === modified file 'sql/mysqld.h' > --- a/sql/mysqld.h 2014-09-26 10:37:19 +0000 > +++ b/sql/mysqld.h 2014-10-07 14:20:37 +0000 > @@ -187,7 +187,6 @@ extern ulong stored_program_cache_size; > extern ulong opt_slave_parallel_threads; > extern ulong opt_slave_domain_parallel_threads; > extern ulong opt_slave_parallel_max_queued; > -extern ulonglong opt_slave_parallel_mode; > extern ulong opt_binlog_commit_wait_count; > extern ulong opt_binlog_commit_wait_usec; > extern my_bool opt_gtid_ignore_duplicates; > > === modified file 'sql/rpl_mi.cc' > --- a/sql/rpl_mi.cc 2014-09-02 12:07:01 +0000 > +++ b/sql/rpl_mi.cc 2014-10-07 14:20:37 +0000 > @@ -40,7 +40,8 @@ Master_info::Master_info(LEX_STRING *con > heartbeat_period(0), received_heartbeats(0), master_id(0), > prev_master_id(0), > using_gtid(USE_GTID_NO), events_queued_since_last_gtid(0), > - gtid_reconnect_event_skip_count(0), gtid_event_seen(false) > + gtid_reconnect_event_skip_count(0), gtid_event_seen(false), > + parallel_mode(SLAVE_PARALLEL_DOMAIN|SLAVE_PARALLEL_GROUPCOMMIT) > { > host[0] = 0; user[0] = 0; password[0] = 0; > ssl_ca[0]= 0; ssl_capath[0]= 0; ssl_cert[0]= 0; > @@ -178,6 +179,7 @@ void init_master_log_pos(Master_info* mi > mi->events_queued_since_last_gtid= 0; > mi->gtid_reconnect_event_skip_count= 0; > mi->gtid_event_seen= false; > + mi->parallel_mode= SLAVE_PARALLEL_DOMAIN|SLAVE_PARALLEL_GROUPCOMMIT; > > /* Intentionally init ssl_verify_server_cert to 0, no option available > */ > mi->ssl_verify_server_cert= 0; > @@ -514,6 +516,22 @@ file '%s')", fname); > else > mi->using_gtid= Master_info::USE_GTID_NO; > } > + else if (!strncmp(buf, STRING_WITH_LEN("parallel_mode="))) > + { > + int val= atoi(buf + (sizeof("parallel_mode=") - 1)); > + mi->parallel_mode= val & (SLAVE_PARALLEL_DOMAIN | > + SLAVE_PARALLEL_GROUPCOMMIT | > + SLAVE_PARALLEL_TRX | > + SLAVE_PARALLEL_WAITING); > + } > + else if (!strncmp(buf, STRING_WITH_LEN("END_MARKER"))) > + { > + /* > + Guard agaist extra left-overs at the end of file, in case a > later > + update causes the file to shrink compared to earlier > contents. > + */ > + break; > + } > } > } > } > @@ -657,7 +675,9 @@ int flush_master_info(Master_info* mi, > my_b_printf(file, > > "%u\n%s\n%s\n%s\n%s\n%s\n%d\n%d\n%d\n%s\n%s\n%s\n%s\n%s\n%d\n%s\n%s\n%s\n%s\n%d\n%s\n%s\n" > "\n\n\n\n\n\n\n\n\n\n\n" > - "using_gtid=%d\n", > + "using_gtid=%d\n" > + "parallel_mode=%u\n" > + "END_MARKER\n", > LINES_IN_MASTER_INFO, > mi->master_log_name, llstr(mi->master_log_pos, lbuf), > mi->host, mi->user, > @@ -666,7 +686,7 @@ int flush_master_info(Master_info* mi, > mi->ssl_cipher, mi->ssl_key, mi->ssl_verify_server_cert, > heartbeat_buf, "", ignore_server_ids_buf, > "", 0, > - mi->ssl_crl, mi->ssl_crlpath, mi->using_gtid); > + mi->ssl_crl, mi->ssl_crlpath, mi->using_gtid, > mi->parallel_mode); > my_free(ignore_server_ids_buf); > err= flush_io_cache(file); > if (sync_masterinfo_period && !err && > > === modified file 'sql/rpl_mi.h' > --- a/sql/rpl_mi.h 2014-09-02 12:07:01 +0000 > +++ b/sql/rpl_mi.h 2014-10-07 14:20:37 +0000 > @@ -75,6 +75,10 @@ class Master_info : public Slave_reporti > return connection_name.str == 0; > } > static const char *using_gtid_astext(enum enum_using_gtid arg); > + bool using_parallel() > + { > + return opt_slave_parallel_threads > 0 && parallel_mode != 0; > + } > > /* the variables below are needed because we can change masters on the > fly */ > char master_log_name[FN_REFLEN+6]; /* Room for multi-*/ > @@ -178,6 +182,12 @@ class Master_info : public Slave_reporti > uint64 gtid_reconnect_event_skip_count; > /* gtid_event_seen is false until we receive first GTID event from > master. */ > bool gtid_event_seen; > + /* > + The parallel replication modes, if any. A combination (binary OR) of > any of > + SLAVE_PARALLEL_DOMAIN, SLAVE_PARALLEL_GROUPCOMMIT, > SLAVE_PARALLEL_TRX, and > + SLAVE_PARALLEL_WAITING. > + */ > + uint32 parallel_mode; > }; > int init_master_info(Master_info* mi, const char* master_info_fname, > const char* slave_info_fname, > > === modified file 'sql/rpl_parallel.cc' > --- a/sql/rpl_parallel.cc 2014-10-03 09:43:23 +0000 > +++ b/sql/rpl_parallel.cc 2014-10-07 14:20:37 +0000 > @@ -1891,6 +1891,7 @@ rpl_parallel::do_event(rpl_group_info *s > bool did_enter_cond= false; > PSI_stage_info old_stage; > > + DBUG_EXECUTE_IF("slave_crash_if_parallel_apply", DBUG_SUICIDE();); > /* Handle master log name change, seen in Rotate_log_event. */ > typ= ev->get_type_code(); > if (unlikely(typ == ROTATE_EVENT)) > @@ -1971,7 +1972,8 @@ rpl_parallel::do_event(rpl_group_info *s > if (typ == GTID_EVENT) > { > Gtid_log_event *gtid_ev= static_cast<Gtid_log_event *>(ev); > - uint32 domain_id= (rli->mi->using_gtid == Master_info::USE_GTID_NO ? > + uint32 domain_id= (rli->mi->using_gtid == Master_info::USE_GTID_NO || > + !(rli->mi->parallel_mode & SLAVE_PARALLEL_DOMAIN) ? > 0 : gtid_ev->domain_id); > if (!(e= find(domain_id))) > { > @@ -2012,7 +2014,7 @@ rpl_parallel::do_event(rpl_group_info *s > { > Gtid_log_event *gtid_ev= static_cast<Gtid_log_event *>(ev); > bool new_gco; > - ulonglong mode= opt_slave_parallel_mode; > + ulonglong mode= rli->mi->parallel_mode; > uchar gtid_flags= gtid_ev->flags2; > group_commit_orderer *gco; > uint8 force_switch_flag; > > === modified file 'sql/rpl_parallel.h' > --- a/sql/rpl_parallel.h 2014-10-03 09:43:23 +0000 > +++ b/sql/rpl_parallel.h 2014-10-07 14:20:37 +0000 > @@ -12,7 +12,11 @@ class Relay_log_info; > struct inuse_relaylog; > > > -/* Bit masks for the values in --slave-parallel-mode. */ > +/* > + Bit masks for the values in --slave-parallel-mode. > + Note that these values cannot be changed - they are stored in > master.info, > + so need to be possible to read back in a different version of the > server. > +*/ > #define SLAVE_PARALLEL_DOMAIN (1ULL << 0) > #define SLAVE_PARALLEL_GROUPCOMMIT (1ULL << 1) > #define SLAVE_PARALLEL_TRX (1ULL << 2) > > === modified file 'sql/share/errmsg-utf8.txt' > --- a/sql/share/errmsg-utf8.txt 2014-07-08 17:38:26 +0000 > +++ b/sql/share/errmsg-utf8.txt 2014-10-07 14:20:37 +0000 > @@ -7111,3 +7111,5 @@ ER_SLAVE_SKIP_NOT_IN_GTID > eng "When using GTID, @@sql_slave_skip_counter can not be used. > Instead, setting @@gtid_slave_pos explicitly can be used to skip to after a > given GTID position." > ER_TABLE_DEFINITION_TOO_BIG > eng "The definition for table %`s is too big" > +ER_INVALID_SLAVE_PARALLEL_MODE > + eng "Invalid use of '%s' option for PARALLEL_MODE in CHANGE > MASTER TO" > > === modified file 'sql/slave.cc' > --- a/sql/slave.cc 2014-09-02 12:07:01 +0000 > +++ b/sql/slave.cc 2014-10-07 14:20:37 +0000 > @@ -625,8 +625,7 @@ int terminate_slave_threads(Master_info* > if (thread_mask & (SLAVE_SQL|SLAVE_FORCE_ALL)) > { > DBUG_PRINT("info",("Terminating SQL thread")); > - if (opt_slave_parallel_threads > 0 && > - mi->rli.abort_slave && mi->rli.stop_for_until) > + if (mi->using_parallel() && mi->rli.abort_slave && > mi->rli.stop_for_until) > { > mi->rli.stop_for_until= false; > mi->rli.parallel.stop_during_until(); > @@ -2576,6 +2575,8 @@ static bool send_show_master_info_header > field_list.push_back(new Item_empty_string("Using_Gtid", > sizeof("Current_Pos")-1)); > field_list.push_back(new Item_empty_string("Gtid_IO_Pos", 30)); > + field_list.push_back(new Item_empty_string("Parallel_Mode", > + sizeof("domain,groupcommit,transactional,waiting")-1)); > if (full) > { > field_list.push_back(new Item_return_int("Retried_transactions", > @@ -2708,8 +2709,7 @@ static bool send_show_master_info_data(T > else > { > idle= mi->rli.sql_thread_caught_up; > - if (opt_slave_parallel_threads > 0 && idle && > - !mi->rli.parallel.workers_idle()) > + if (mi->using_parallel() && idle && > !mi->rli.parallel.workers_idle()) > idle= false; > } > if (idle) > @@ -2786,13 +2786,34 @@ static bool send_show_master_info_data(T > protocol->store(mi->ssl_ca, &my_charset_bin); > // Master_Ssl_Crlpath > protocol->store(mi->ssl_capath, &my_charset_bin); > + // Using_Gtid > protocol->store(mi->using_gtid_astext(mi->using_gtid), > &my_charset_bin); > + // Gtid_IO_Pos > { > char buff[30]; > String tmp(buff, sizeof(buff), system_charset_info); > mi->gtid_current_pos.to_string(&tmp); > protocol->store(tmp.ptr(), tmp.length(), &my_charset_bin); > } > + // Parallel_Mode > + { > + /* Note how sizeof("domain") has room for "domain," due to traling > 0. */ > + char buf[sizeof("domain") + sizeof("groupcommit") + > + sizeof("transactional") + sizeof("waiting") + 1]; > + char *p= buf; > + uint32 mode= mi->parallel_mode; > + if (mode & SLAVE_PARALLEL_DOMAIN) > + p= strmov(p, "domain,"); > + if (mode & SLAVE_PARALLEL_GROUPCOMMIT) > + p= strmov(p, "groupcommit,"); > + if (mode & SLAVE_PARALLEL_TRX) > + p= strmov(p, "transactional,"); > + if (mode & SLAVE_PARALLEL_WAITING) > + p= strmov(p, "waiting,"); > + if (p != buf) > + --p; // Discard last ',' > + protocol->store(buf, p-buf, &my_charset_bin); > + } > if (full) > { > protocol->store((uint32) mi->rli.retried_trans); > @@ -3521,7 +3542,7 @@ static int exec_relay_log_event(THD* thd > > update_state_of_relay_log(rli, ev); > > - if (opt_slave_parallel_threads > 0) > + if (rli->mi->using_parallel()) > { > int res= rli->parallel.do_event(serial_rgi, ev, event_size); > if (res >= 0) > @@ -4667,7 +4688,7 @@ log '%s' at position %s, relay log '%s' > } > } > > - if (opt_slave_parallel_threads > 0) > + if (mi->using_parallel()) > rli->parallel.wait_for_done(thd, rli); > > /* Thread stopped. Print the current replication position to the log */ > @@ -4693,7 +4714,7 @@ log '%s' at position %s, relay log '%s' > (We want the first one to be before the printout of stop position to > get the correct position printed.) > */ > - if (opt_slave_parallel_threads > 0) > + if (mi->using_parallel()) > rli->parallel.wait_for_done(thd, rli); > > /* > @@ -6314,7 +6335,7 @@ static Log_event* next_event(rpl_group_i > llstr(my_b_tell(cur_log),llbuf1), > llstr(rli->event_relay_log_pos,llbuf2))); > DBUG_ASSERT(my_b_tell(cur_log) >= BIN_LOG_HEADER_SIZE); > - DBUG_ASSERT(opt_slave_parallel_threads > 0 || > + DBUG_ASSERT(rli->mi->using_parallel() || > my_b_tell(cur_log) == rli->event_relay_log_pos); > } > #endif > > === modified file 'sql/sql_lex.h' > --- a/sql/sql_lex.h 2014-08-07 16:06:56 +0000 > +++ b/sql/sql_lex.h 2014-10-07 14:20:37 +0000 > @@ -218,11 +218,18 @@ struct LEX_MASTER_INFO > uint port, connect_retry; > float heartbeat_period; > /* > + Modes of parallel replication enabled, if any. A combination (binary > OR) of > + any of SLAVE_PARALLEL_DOMAIN, SLAVE_PARALLEL_GROUPCOMMIT, > + SLAVE_PARALLEL_TRX, and SLAVE_PARALLEL_WAITING. > + */ > + uint32 repl_parallel_mode; > + /* > Enum is used for making it possible to detect if the user > changed variable or if it should be left at old value > */ > enum {LEX_MI_UNCHANGED, LEX_MI_DISABLE, LEX_MI_ENABLE} > - ssl, ssl_verify_server_cert, heartbeat_opt, > repl_ignore_server_ids_opt; > + ssl, ssl_verify_server_cert, heartbeat_opt, > repl_ignore_server_ids_opt, > + repl_parallel_mode_opt; > enum { > LEX_GTID_UNCHANGED, LEX_GTID_NO, LEX_GTID_CURRENT_POS, > LEX_GTID_SLAVE_POS > } use_gtid_opt; > @@ -241,10 +248,11 @@ struct LEX_MASTER_INFO > pos= relay_log_pos= server_id= port= connect_retry= 0; > heartbeat_period= 0; > ssl= ssl_verify_server_cert= heartbeat_opt= > - repl_ignore_server_ids_opt= LEX_MI_UNCHANGED; > + repl_ignore_server_ids_opt= repl_parallel_mode_opt= > LEX_MI_UNCHANGED; > gtid_pos_str.length= 0; > gtid_pos_str.str= NULL; > use_gtid_opt= LEX_GTID_UNCHANGED; > + repl_parallel_mode= 0; > } > }; > > > === modified file 'sql/sql_repl.cc' > --- a/sql/sql_repl.cc 2014-08-07 16:06:56 +0000 > +++ b/sql/sql_repl.cc 2014-10-07 14:20:37 +0000 > @@ -3434,6 +3434,9 @@ bool change_master(THD* thd, Master_info > lex_mi->relay_log_name || lex_mi->relay_log_pos) > mi->using_gtid= Master_info::USE_GTID_NO; > > + if (lex_mi->repl_parallel_mode_opt != LEX_MASTER_INFO::LEX_MI_UNCHANGED) > + mi->parallel_mode= lex_mi->repl_parallel_mode; > + > /* > If user did specify neither host nor port nor any log name nor any log > pos, i.e. he specified only user/password/master_connect_retry, he > probably > > === modified file 'sql/sql_yacc.yy' > --- a/sql/sql_yacc.yy 2014-09-03 14:24:31 +0000 > +++ b/sql/sql_yacc.yy 2014-10-07 14:20:37 +0000 > @@ -1130,6 +1130,7 @@ bool my_yyoverflow(short **a, YYSTYPE ** > %token DIV_SYM > %token DOUBLE_SYM /* SQL-2003-R */ > %token DO_SYM > +%token DOMAIN_SYM > %token DROP /* SQL-2003-R */ > %token DUAL_SYM > %token DUMPFILE > @@ -1194,6 +1195,7 @@ bool my_yyoverflow(short **a, YYSTYPE ** > %token GRANT /* SQL-2003-R */ > %token GRANTS > %token GROUP_SYM /* SQL-2003-R */ > +%token GROUPCOMMIT_SYM > %token GROUP_CONCAT_SYM > %token GT_SYM /* OPERATOR */ > %token HANDLER_SYM > @@ -1376,6 +1378,7 @@ bool my_yyoverflow(short **a, YYSTYPE ** > %token PACK_KEYS_SYM > %token PAGE_SYM > %token PAGE_CHECKSUM_SYM > +%token PARALLEL_MODE_SYM > %token PARAM_MARKER > %token PARSER_SYM > %token PARSE_VCOL_EXPR_SYM > @@ -1604,6 +1607,7 @@ bool my_yyoverflow(short **a, YYSTYPE ** > %token VIEW_SYM /* SQL-2003-N */ > %token VIRTUAL_SYM > %token WAIT_SYM > +%token WAITING_SYM > %token WARNINGS > %token WEEK_SYM > %token WEIGHT_STRING_SYM > @@ -2244,6 +2248,14 @@ rule: <-- starts at col 1 > { > Lex->mi.repl_ignore_server_ids_opt= > LEX_MASTER_INFO::LEX_MI_ENABLE; > } > + | PARALLEL_MODE_SYM EQ '(' ')' > + { > + Lex->mi.repl_parallel_mode_opt= > LEX_MASTER_INFO::LEX_MI_DISABLE; > + } > + | PARALLEL_MODE_SYM EQ '(' parallel_mode_list ')' > + { > + Lex->mi.repl_parallel_mode_opt= > LEX_MASTER_INFO::LEX_MI_ENABLE; > + } > | > master_file_def > ; > @@ -2260,6 +2272,52 @@ rule: <-- starts at col 1 > insert_dynamic(&Lex->mi.repl_ignore_server_ids, (uchar*) > &($1)); > } > > +parallel_mode_list: > + parallel_mode_option > + | parallel_mode_list ',' parallel_mode_option > + ; > + > +parallel_mode_option: > + DOMAIN_SYM > + { > + if (Lex->mi.repl_parallel_mode & SLAVE_PARALLEL_DOMAIN) > + { > + my_error(ER_INVALID_SLAVE_PARALLEL_MODE, MYF(0), > "domain"); > + MYSQL_YYABORT; > + } > + Lex->mi.repl_parallel_mode|= SLAVE_PARALLEL_DOMAIN; > + } > + | GROUPCOMMIT_SYM > + { > + if (Lex->mi.repl_parallel_mode & > + (SLAVE_PARALLEL_GROUPCOMMIT|SLAVE_PARALLEL_TRX)) > + { > + my_error(ER_INVALID_SLAVE_PARALLEL_MODE, MYF(0), > "groupcommit"); > + MYSQL_YYABORT; > + } > + Lex->mi.repl_parallel_mode|= SLAVE_PARALLEL_GROUPCOMMIT; > + } > + | TRANSACTIONAL_SYM > + { > + if (Lex->mi.repl_parallel_mode & > + (SLAVE_PARALLEL_GROUPCOMMIT|SLAVE_PARALLEL_TRX)) > + { > + my_error(ER_INVALID_SLAVE_PARALLEL_MODE, MYF(0), > "transactional"); > + MYSQL_YYABORT; > + } > + Lex->mi.repl_parallel_mode|= SLAVE_PARALLEL_TRX; > + } > + | WAITING_SYM > + { > + if (Lex->mi.repl_parallel_mode & SLAVE_PARALLEL_WAITING) > + { > + my_error(ER_INVALID_SLAVE_PARALLEL_MODE, MYF(0), > "waiting"); > + MYSQL_YYABORT; > + } > + Lex->mi.repl_parallel_mode|= SLAVE_PARALLEL_WAITING; > + } > + > + > master_file_def: > MASTER_LOG_FILE_SYM EQ TEXT_STRING_sys > { > @@ -14098,6 +14156,7 @@ user: user_maybe_role > | DISABLE_SYM {} > | DISCARD {} > | DISK_SYM {} > + | DOMAIN_SYM {} > | DUMPFILE {} > | DUPLICATE_SYM {} > | DYNAMIC_SYM {} > @@ -14131,6 +14190,7 @@ user: user_maybe_role > | GET_FORMAT {} > | GRANTS {} > | GLOBAL_SYM {} > + | GROUPCOMMIT_SYM {} > | HASH_SYM {} > | HARD_SYM {} > | HOSTS_SYM {} > @@ -14221,6 +14281,7 @@ user: user_maybe_role > | ONLY_SYM {} > | PACK_KEYS_SYM {} > | PAGE_SYM {} > + | PARALLEL_MODE_SYM {} > | PARTIAL {} > | PARTITIONING_SYM {} > | PARTITIONS_SYM {} > @@ -14336,6 +14397,7 @@ user: user_maybe_role > | VALUE_SYM {} > | WARNINGS {} > | WAIT_SYM {} > + | WAITING_SYM {} > | WEEK_SYM {} > | WEIGHT_STRING_SYM {} > | WORK_SYM {} > > === modified file 'sql/sys_vars.cc' > --- a/sql/sys_vars.cc 2014-09-19 13:25:37 +0000 > +++ b/sql/sys_vars.cc 2014-10-07 14:20:37 +0000 > @@ -1834,31 +1834,11 @@ static Sys_var_ulong Sys_slave_parallel_ > VALID_RANGE(0,2147483647), DEFAULT(131072), BLOCK_SIZE(1)); > > > -static const char *slave_parallel_mode_names[] = { > - "domain", "groupcommit", "transactional", "waiting", NULL > -}; > - > -static Sys_var_set Sys_slave_parallel_mode( > - "slave_parallel_mode", > - "Controls what transactions are applied in parallel when using " > - "--slave-parallel-threads. Syntax: > slave_paralle_mode=value[,value...], " > - "where \"value\" could be one or more of: \"domain\", to apply " > - "different replication domains in parallel; \"groupcommit\", to > apply " > - "in parallel transactions that group-committed together on the > master; " > - "\"transactional\", to optimistically try to apply all > transactional " > - "DML in parallel; and \"waiting\" to extend \"transactional\" to " > - "even transactions that had to wait on the master.", > - GLOBAL_VAR(opt_slave_parallel_mode), CMD_LINE(REQUIRED_ARG), > - slave_parallel_mode_names, > - DEFAULT(SLAVE_PARALLEL_DOMAIN | > - SLAVE_PARALLEL_GROUPCOMMIT)); > - > - > static Sys_var_bit Sys_replicate_allow_parallel( > "replicate_allow_parallel", > "If set when a transaction is written to the binlog, that > transaction " > - "is allowed to replicate in parallel on a slave where " > - "slave_parallel_mode is set to \"transactional\". Can be cleared > for " > + "is allowed to replicate in parallel on a slave where > parallel_mode " > + "is set to \"transactional\" (in CHANGE MASTER). Can be cleared > for " > "transactions that are likely to cause a conflict if replicated in > " > "parallel, to avoid unnecessary rollback and retry.", > SESSION_ONLY(option_bits), NO_CMD_LINE, OPTION_RPL_ALLOW_PARALLEL, > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits >

2 1

[Maria-developers] MDEV-6429 (storage engine API for parallel replication) possibly a requirement for MDEV-6676 (speculative parallel replication)
by Kristian Nielsen 06 Oct '14

06 Oct '14

Hi Serg, Earlier this year we discussed MDEV-6429, which is to implement a proper storage engine API for a feature used in parallel replication. This is where InnoDB reports to the replication layer whenever transaction T1 needs to wait for T2; parallel replication uses this to detect and resolve deadlocks caused by enforced commit order. In 10.0, we decided to not do a "proper" API, instead just modify InnoDB to call into the server layer directly. In 10.0, the need for this API/feature is somewhat InnoDB-specific, it is not known whether any of the other transactional storage engines would need to do anything similar for parallel replication to work. If we implement MDEV-6676, however, then the need for the API becomes more critical. Because MDEV-6676 absolutely only works if we have a mechanism to detect conflicts that cause a deadlock with commit order, as was implemented for InnoDB. If we were to attempt to enable speculative parallel replication for eg. Tokutek transactions, without implementing the necessary check in Tokutek, then the slave would quickly end up hanging. Thinking more on this, I suppose what we really need is what you requested also during the original discussion: A true optional feature for storage engines to implement this additional check. And a way for the server layer to see if a storage engine has the implementation. If it does, then we can try to run its transactions in parallel as per MDEV-6676. If not, we will not try, and things will still work correctly (just without the parallelisation optimisation). Maybe for the first version, the best thing to do is to just hard-code a check for the InnoDB storage engine. This will allow us to get real-world testing to see if the new speculative parallel replication feature is interesting to any users. Then if it is, and there turns out to be interest from other storage engine venders (such as Tokutek) to also support speculative parallel replication, we can work with them to implement a proper API. This approach would also allow to consider the needs of a different storage engine when designing the API, which makes it much more likely that the API will be of general use and not too tied to InnoDB internals. What do you think? - Kristian.

1 0

[Maria-developers] MariaDB Debian packaging: pending merge to upstream
by Otto Kekäläinen 03 Oct '14

03 Oct '14

Hello! Now when the rehearsal is done using simple spelling error fix patches, I now posted the first big merge request about my 5.5/debian upon the upstream 5.5 branch. Issue at https://mariadb.atlassian.net/browse/MDEV-6284 Merge request at https://code.launchpad.net/~otto/maria/maria-fix-debpkg-5.5/+merge/221576 I also have these old merge request still open: https://code.launchpad.net/~otto/maria/maria-fix-debian-copyright-trunk/+me… https://code.launchpad.net/~otto/maria/maria-fix-debian-copyright-5.5/+merg… The sooner somebody has can merge these the less likely it is that there will be merge conflicts.. Thanks! -- Check out our blog at http://seravo.fi/blog and follow @ottokekalainen

2 6

Re: [Maria-developers] Fwd: [Commits] Rev 3897: bzr merge -r4123..4144 codership/5.6 in lp:~maria-captains/maria/maria-10.0-galera
by Philip Stoev 01 Oct '14

01 Oct '14

Hello, Thanks for taking the time to review this. Your concerns are spot-on. == modified file 'mysql-test/mysql-test-run.pl' --- a/mysql-test/mysql-test-run.pl 2014-09-29 00:43:56 +0000 +++ b/mysql-test/mysql-test-run.pl 2014-09-30 22:06:15 +0000 @@ -3189,6 +3189,9 @@ mtr_report(" - adding wsrep, galera to default test suites"); push @DEFAULT_SUITES, qw(wsrep galera); + # ADD scripts to $PATH to that wsrep_sst_* can be found + $ENV{'PATH'} = $ENV{'PATH'}.':'.$basedir.'/ scripts'; > Is this the right place to do this ? Serg already commented about the fact > that we should not "temper" mtr globally, instead only when galera tests > are run, I just wonder because at scripts directory contain other than > wsrep scripts, does this affect existing tests... Yes, you are right, I reverted this change and moved it into suite/galera/galera2nodes.cnf, which now looks like this: [ENV] PATH=/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:@ENV.MYSQL_BINDIR/scripts Unfortunately, in the context of .cnf files, MTR does not allow to simply append to the existing $PATH, so I have to reinitialize it from scratch. === modified file 'mysql-test/suite/galera/ galera_2nodes.cnf' --- a/mysql-test/suite/galera/galera_2nodes.cnf 2014-08-26 20:23:56 +0000 +++ b/mysql-test/suite/galera/galera_2nodes.cnf 2014-09-30 22:06:15 +0000 @@ -2,22 +2,36 @@ !include include/default_mysqld.cnf [mysqld.1] +innodb_file_per_table binlog-format=row +innodb_file_format=Barracuda +performance_schema=OFF + wsrep_provider=(a)ENV.WSREP_PROVIDER wsrep_cluster_address='gcomm://' wsrep_provider_options='base_port=(a)mysqld.1.#galera_port' wsrep_sst_receive_address='127.0.0.1:@mysqld.1.#sst_port' # enforce read-committed characteristics across the cluster wsrep_causal_reads=ON +wsrep_sync_wait = 7 +wsrep_node_address=127.0.0.1 +wsrep_node_incoming_address=127.0.0.1 > Why we need to have innodb_file_per_table=1, innodb_file_format=Barracuda > and performance_schema=OFF ? I have removed those options. performance_schema was OFF because there was an assertion that affected all the tests. Secondly, why wsrep_sync_wait=7 (can't these different numbers have more meaningful values i.e. enum ?) This is required to ensure that all statements see the latest state of the database, allowing for deterministic tests. With respect to using a bitmask vs. an enum, please feel free to open an issue in LaunchPad. Philip Stoev

1 0

Re: [Maria-developers] [Commits] Rev 3897: bzr merge -r4123..4144 codership/5.6 in lp:~maria-captains/maria/maria-10.0-galera
by Sergei Golubchik 01 Oct '14

01 Oct '14

Hi, Jan! On Oct 01, Jan Lindström wrote: > Hi Nirbhay, Philip > > First of all, it is good to see that we have more galera tests. Few > questions: > > == modified file 'mysql-test/mysql-test-run.pl' > --- a/mysql-test/mysql-test-run.pl 2014-09-29 00:43:56 +0000 > +++ b/mysql-test/mysql-test-run.pl 2014-09-30 22:06:15 +0000 > @@ -3189,6 +3189,9 @@ > mtr_report(" - adding wsrep, galera to default test suites"); > push @DEFAULT_SUITES, qw(wsrep galera); > > + # ADD scripts to $PATH to that wsrep_sst_* can be found > + $ENV{'PATH'} = $ENV{'PATH'}.':'.$basedir.'/scripts'; > > Is this the right place to do this ? Serg already commented about the fact > that we should not "temper" mtr globally, instead only when galera tests > are run, I just wonder because at scripts directory contain other than > wsrep scripts, does this affect existing tests... And, just to note, in 10.1 I did it somewhat differently: my ($path) = grep { -f "$_/wsrep_sst_rsync"; } "$::bindir/scripts", $::path_client_bindir; return "No SST scripts" unless $path; $ENV{PATH}="$path:$ENV{PATH}"; Note the differences: - I check whether a script really exists - I look in two locations - same paths where mtr looks for mysqlhotcopy - the path is added to the beginning of the PATH, because we want to find our scripts before any other similar scripts that might happen to be in the PATH because of the existing MariaDB-Galera installation Regards, Sergei

1 0

[Maria-developers] Aria bug with query cache - wrong result and wrong table invalidation
by Roberto Spadim 30 Sep '14

30 Sep '14

Hi guys, could someone take a look at MDEV-6817? i'm testing a stress test with aria and row_format=page the problem is, running many php>while(1){ mysql>select max(column) from table; php> get the value, sum+1 mysql>insert into table (the value of column+1) php>} i got some problems... the value of the first max(column) sometimes return an old value in others words, the query isn't consistent, or the table invalidation code is wrong i got a value 296 at query cache if i change the autocommit from 1 to 0, i got a value of 297, i could use SQL_NO_CACHE and got the same 297 value the problem only occur with row_format=page -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

Re: [Maria-developers] Patch: CREATE [[NO] FORCE] VIEW Links
by James Briggs 30 Sep '14

30 Sep '14

Hi folks. Thank you Elena for commenting on MDEV-6709 "Disambiguating "CREATE FORCE VIEW view1 AS SELECT * FROM TABLE1" "As I understand, your patch hasn't been accepted/merged into the main tree yet. Why don't you apply the bugfix in your tree and resubmit the patch?" 1) MDEV-6709 is a related but separate issue from my FORCE patch, applying to MDEV-356 and others. I think it should be a separate patch. 2) What would you like me to do next to get my original patch accepted? 3) Who is the current maintainer for views? I feel that the view features are only 50% implemented in MariaDB and need some design effort. Thanks, James Briggs. -- Cassandra/MySQL DBA. Available in San Jose area or remote. ________________________________ From: James Briggs <actionmessage(a)yahoo.com> To: Maria-developers <maria-developers(a)lists.launchpad.net> Sent: Monday, September 8, 2014 2:52 PM Subject: Patch: CREATE [[NO] FORCE] VIEW Links Hi List. I mailed two patches to the list to implement CREATE [[NO] FORCE] plus tests this weekend. It looks like the mailing list software filters out long posts, so here are links to the list archives: Patch: CREATE [[NO] FORCE] VIEW Options https://lists.launchpad.net/maria-developers/msg07681.html Patch: CREATE VIEW test program https://lists.launchpad.net/maria-developers/msg07682.htm Thanks, James Briggs. -- Available for MySQL/NoSQL DBA/Programming in San Jose area or remote.

1 0

[Maria-developers] [GSoC2014] Self-tuning cost coefficients
by Anshu Avinash 28 Sep '14

28 Sep '14

Hi all, I have written a documentation for framework which I have created for gathering and calculating cost coefficients for optimizer, as part of my GSoC project. It is available at http://igniting.in/gsoc2014/2014/08/20/documentation/. Any suggestions/reviews are welcome. Regards Anshu Avinash

3 3

[Maria-developers] MariaDB 10.0.14 got rid of replication safety warning without warning in release notes?
by Pavel Ivanov 26 Sep '14

26 Sep '14

Hi, I see that in MariaDB 10.0.14 Michael disabled replication safety warning for the case of INSERT ON DUPLICATE KEY UPDATE with several unique keys with a comment "We should assume that the store engine will report the first duplicate key for this case." http://bazaar.launchpad.net/~maria-captains/maria/10.0/revision/4400.1.3 What does that mean? Which key is "the first duplicate key" and why MariaDB relies on an engine behavior which AFAIK is not guaranteed? And why this serious change in behavior wasn't mentioned in the Release Notes? On the related note Release Notes mention the error log flood protection. Why it's not behind flag? How should I disable the feature and get back to the old behavior? Thank you, Pavel

2 2

Re: [Maria-developers] [Commits] 236cc6c: wsrep-related changes: removed some unnecessary files & minor modifications.
by Sergei Golubchik 26 Sep '14

26 Sep '14

Hi, Nirbhay! On Sep 25, nirbhay(a)skysql.com wrote: > revision-id: 236cc6cd49f250a43f0f6a794d5052207b2d9348 > parent(s): a756ac61900e28448776427e6ebbb0ef6f48d5cc > committer: Nirbhay Choubey > branch nick: server > timestamp: 2014-09-25 20:59:15 -0400 > message: > > wsrep-related changes: removed some unnecessary files & minor modifications. Nirbhay, I've ask your few times on irc and email *not* to do that. I already have all these changes in my tree, now I'll need to merge that. It adds no value, only wasted double work and wasted time for useless merges. Please don't do changes that I've already done! Regards, Sergei

1 0

Re: [Maria-developers] More upgrades needed for work-amd64-valgrind
by Sergey Petrunia 25 Sep '14

25 Sep '14

Hi Jani, On Thu, Sep 25, 2014 at 02:22:48PM +0300, Jani Tolonen wrote: > Hi Sergey, > > cmake version was just one minor version too old. I upgraded it and > libstdc++ and according to cmake documentation the error should now be > gone. Please let me know if there are still problems. It seems like not - valgrind builder has has built 10.1 tree and is now running tests (for the first time in ... :). Thank you for your help! BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] More upgrades needed for work-amd64-valgrind
by Sergey Petrunia 25 Sep '14

25 Sep '14

Hi Jani, Thanks. Please let me know when cmake upgrade is finished. On Wed, Sep 24, 2014 at 04:07:51PM +0300, Jani Tolonen wrote: > Hi Sergei, > > Ok, I will check if I can get cmake to work a bit later tonight. > > BR, > Jani > > On 24 September 2014 14:48, Sergey Petrunia <sergey(a)mariadb.com> wrote: > > > Hello Jani, > > > > Thanks for getting git to work for work-amd64-valgrind host. > > > > I am looking at > > http://buildbot.askmonty.org/buildbot/grid?branch=10.1 > > > > and see that "git update" step succeeds. However, the next step fails: > > > > > > http://buildbot.askmonty.org/buildbot/builders/work-amd64-valgrind/builds/6… > > > > it is probably because of cmake being too old. > > > > Could you please update cmake on work-amd64-valgrind, also? > > > > BR > > Sergei > > -- > > Sergei Petrunia, Software Developer > > MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog > > > > > > -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] BUG - MDEV 6781
by Roberto Spadim 25 Sep '14

25 Sep '14

Hi guys, i found a bug at query cache and sent a patch to solve it https://mariadb.atlassian.net/browse/MDEV-6781 https://github.com/rspadim/server/commit/5a5e47bf4141ebc3584ed365173dde87b7… there's a pull request at 10.1 github (that's only a line removing a +1 at database name length) please review the patch that was only found cause i'm using qc_info plugin from mdev-4682 thanks :) -- Roberto Spadim

1 0

[Maria-developers] OQGRAPH/others engine and query cache
by Roberto Spadim 24 Sep '14

24 Sep '14

Hi guys i was checking OQGRAPH limitations with query cache it can't use query cache cause we don't know when data base change, that's ok right? i think that's ok too some others engines have this 'problem' too the first answer is, since oqgraph can't know when data table changes, we can't cache oqgraph tables... i think that's right BUT... query cache 'save' table used in query cache, for example: select * from oq_graph this will include "select * from oq_graph" at query cache using table 'oq_graph' instead of only using 'oq_graph' we could include 'oq_graph' AND 'data_table' even when we 'only' use oq_graph at FROM part this is more interesting than turning query cache off to oq_graph, and this remove the problem of "the data table don't tell oqgraph when we need to invalidate query cache" and that's right! but since data_table changed it will invalidate all queries using it, including oq_graph queries =) check if i'm not wrong, i can try a patch with it if we don't have others query cache related problems (for example query cache don't support tables outside query) -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

[Maria-developers] More upgrades needed for work-amd64-valgrind
by Sergey Petrunia 24 Sep '14

24 Sep '14

Hello Jani, Thanks for getting git to work for work-amd64-valgrind host. I am looking at http://buildbot.askmonty.org/buildbot/grid?branch=10.1 and see that "git update" step succeeds. However, the next step fails: http://buildbot.askmonty.org/buildbot/builders/work-amd64-valgrind/builds/6… it is probably because of cmake being too old. Could you please update cmake on work-amd64-valgrind, also? BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Maria-discuss] MariaDB encryption
by Jonas Oreland 23 Sep '14

23 Sep '14

Hi again, > What is the type of license of your code? I asked internally about license, and it seems like we releasing dual gpl2/apache licensed code. > I would like to know, which interfaces from maria-DB you are using. I don't 100% understand the question. We didn't write any actual encryption code, but used the one provided in openssl. Other than that, we didn't really "use interfaces", but rather added/modified functionality/interfaces here and there. Can you be more specific ? /Jonas On Sat, Jun 7, 2014 at 11:20 PM, Elmar Eperiesi-Beck <elmar(a)eperiesi-beck.de > wrote: > Hi! > We (eperi) would be glad to do a joined work with Google. > Our solution works with MS-SQL, Oracle and other DBs and we are currently > porting it to MariaDB - and - as Monty said - its never to late to put some > sources together and make the best for the open source community. > > What is the type of license of your code? > > Jonas, I am looking forward to connect to you directly. > > Regards > Elmar > > Hi! > > > Hi Jonas, > > (same Jonas we know from NDBCLUSTER? :-) Good to see you again) > > > > On 6 Jun 2014, at 02:31, Jonas Oreland <jonaso(a)google.com> wrote: > > > >> Hi there, > >> I read this blog post > >> > http://monty-says.blogspot.com/2014/05/for-your-eyes-only-or-adding-better.… > >> and wanted to inform you that we at Google has developed > on-disk/block-level encryption for Innodb, aria (as used by temporary > tables), binlogs and temp-files. > >> The code is not yet published, but we expect it to be within a few > weeks or so. > >> We (of course?) think that it would be better if you instead of > developing new code > >> spent the time testing/reviewing ours. > > We are out course happy to do this! > > >> I'm happy to answer questions on the topic, > >> and will let you know once we've published it. > > The main question I have about the Innodb encryption is if it based on the > compression code we did for fusion-io? > The idea we had on our side was that by using the new compression hooks we > could add encryption with very little changes to the Innodb code. > Looking forward to when you are ready to publish the code so we can > discuss your changes in detail. > > > This is great news! > > > > From what I gather, from Monty's blog post (and a 1:1 we had some time > back), this is something done by a partner/external company that has a > mostly OSS solution, that we should integrate into 10.1 > > Yes, that's correct. It I would have known that Google was working on > encryption I would have included them in my discussions with eperi. > Fortunately it's not yet too late to do this. > I am sure eperi would like to work on the Google code as a base! > > > That said, Google's release of something that works for InnoDB, Aria, > binlogs, temp files (and presumably not too hard to add for MyISAM) is > something we should definitely review and target for 10.1 > > Yes! > > Regards, > Monty > > >

3 21

[Maria-developers] MDEV-4682
by Roberto Spadim 23 Sep '14

23 Sep '14

Hi guys, could check MDEV-4682? it's done to review thanks -- Roberto Spadim

2 2

[Maria-developers] Switching on use of components in Jira
by Rasmus Johansson 23 Sep '14

23 Sep '14

Hello all, I will later today switch on the usage of Components in Jira. Components are a way of grouping issues inside a project. The idea is to map every issue in Jira, being it a bug or a new feature to a component which describes to what area of MariaDB the issue belongs. For example a bug in Aria would be marked with the component "Storage Engine - Aria". Currently there are 37 components defined in Jira that should cover many areas, but I'm sure there are a few missing. There is one generic to which you can assign issues that doesn't fall into any other category, called OTHER. I'll include the list of components here in the end of this email. * What do you need to do? * The Component field will be a mandatory field when closing an issue. Therefore if a component (or several components) haven't been specified earlier for an issue, you'll have to choose at least one when closing an issue. The field has autocompletion so if you start typing for example "Galera" you'll be presented with the alternatives "Galera", "Galera Arbitrator garbd", and "Galera SST". If you don't find a suitable component then choose the component called "OTHER". * What is the benefit of using components? * It's a way of organizing issues inside a project, so that you easily can pull out all issues related to a certain area of MariaDB, e.g. if I wanted to know all bug fixes for replication I would click on the Replication component and get a list. There is the possibility to set up automatic assignees based on components and components can be used in filters and reports throughout Jira. I hope this won't be a burden. All feedback is of course welcome. BR, Rasmus * Components in the MariaDB Server project in JIRA * Data Definition - Alter Table Data Definition - Procedure Data Definition - Temporary Data Manipulation - Insert Delayed Data Manipulation - Subquery Dynamic Columns Events Full-text Search Galera Galera Arbitrator garbd Galera SST GIS Locale Settings OTHER Platform Debian Platform FreeBSD Platform Power Platform Windows Plugin - Audit Plugins Query Cache Replication Show Profile sql_error_log SSL Storage Engine - Aria Storage Engine - Cassandra Storage Engine - Connect Storage Engine - InnoDB Storage Engine - OQGRAPH Storage Engine - Spider Storage Engine - XtraDB Storage Enging - TokuDB Time zones Triggers Views XML Functions

3 4

[Maria-developers] Close some MDEV
by Roberto Spadim 22 Sep '14

22 Sep '14

Hi guys, could close these mdev: 1)MDEV-4599 (it will be implemented at mdev-4584) 2)MDEV-4581 (it will be implemented at mdev-4682) Please check this mdev-4682 too, i sent a patch via github, there's an option that should be implemented at compile time to turn stats on/off i didn't checked how to implement it yet, if anyone could help me :) it must set a macro #define QUERY_CACHE_QC_INFO_PLUGIN 1 at "configure" time, maybe via cmake gui or something like it maybe i should change QUERY_CACHE_QC_INFO_PLUGIN to QUERY_CACHE_INCLUDE_STATS to easier understand of macro definitions, help is wellcome here -- Roberto Spadim

1 0

Re: [Maria-developers] [Commits] Rev 4376: MDEV-6676: Speculative parallel replication in http://bazaar.launchpad.net/~maria-captains/maria/10.0
by Jonas Oreland 22 Sep '14

22 Sep '14

> > Intermediate commit. Patch is far from complete, but this small patch was > nevertheless sufficient to be able to sysbench-0.4 OLTP with full > parallelisation. > "full parallelisation" does that mean that X threads on master make slave achieve k*X higher throughput ? /Jonas

4 16

[Maria-developers] Need suggestions for benchmarking MDEV-6676 parallel replication
by Kristian Nielsen 22 Sep '14

22 Sep '14

Hi Axel, I'd like to ask your advice on some benchmarking of MDEV-6676. MDEV-6676 is about extending in-order parallel replication to be significantly more aggressive, and potentially get a _lot_ more parallelism on the slave. The basic idea is to just try to run _everything_ in parallel. If there is a conflict, the code will roll back the erroneous transaction and retry it, this time not in parallel. I have run a basic sysbench 0.5 OLTP on my laptop, and results were promising. However, I need something better than that. My main problem is that often when I run write-load benchmarks, the bottleneck is InnoDB log write or purge, or something like that. So the load does not really scale with multiple threads, even on the master, and parallel replication is not really that useful even if perfectly implemented. So do you have a suggestion for a write-load benchmark that scales well for medium concurrency, like 8-32 threads or something like that? And for a machine on which to run it? The best benchmark I think would be one that is I/O bound on a working set that is not in memory, on a machine with a disk subsystem that has good throughput at high concurrency. Because this should resemble the real-world use cases where parallel replication is most needed. Do you have any experience on running such benchmarks, and do we have any machines that could be used? Often the benchmarks I see in blogs are in-memory... Or do you have another suggestion? Thanks, - Kristian.

1 0

[Maria-developers] MDEV-4682 revision
by Roberto Spadim 20 Sep '14

20 Sep '14

hi guys, i sent some patch about qc_info plugin ( https://mariadb.atlassian.net/browse/MDEV-4682) via pull request at github (https://github.com/MariaDB/server/pull/6) could anyone check if it's ok? i included test file changes there's only one thing todo, that's include/not include statistics inside query cache, since it increase memory use, i think could be nice at runtime change this variable and include statistics on/off (more info inside mdev) -- Roberto Spadim

1 0

[Maria-developers] doubt about tests files
by Roberto Spadim 20 Sep '14

20 Sep '14

hi guys! i git cloned the last verison of mariadb from github, but i didn't found test files is it outside mariadb github source? -- Roberto Spadim

1 1

Re: [Maria-developers] [Commits] c8496e4: %B
by Roberto Spadim 19 Sep '14

19 Sep '14

working =P 2014-09-19 18:04 GMT-03:00 Igor Babaev <igor(a)askmonty.org>: > revision-id: c8496e48c4b7c35e9702f98daa19cdb4ef190097 > parent(s): 0486b3b796ad7ec597bff775da248096731c58d5 > committer: Igor Babaev > branch nick: 10.1-test-commit > timestamp: 2014-09-19 13:47:43 -0700 > message: > > %B > --- > sql/opt_range.cc | 2 ++ > 1 files changed, 2 insertions(+), 0 deletions(-) > > diff --git a/sql/opt_range.cc b/sql/opt_range.cc > index 27e20c0..c35bde6 100644 > --- a/sql/opt_range.cc > +++ b/sql/opt_range.cc > @@ -14,6 +14,8 @@ > along with this program; if not, write to the Free Software > Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 > USA */ > // test commit in git > +test 2 > + > /* > TODO: > Fix that MAYBE_KEY are stored in the tree so that we can detect use > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits > -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

[Maria-developers] Free Software Foundation address
by Matej Mužila 17 Sep '14

17 Sep '14

Hi, the address of the Free Software Foundation has changed and I think itshould be changed in sources too. Nowdays the correct one is: Free Software Foundation 51 Franklin Street, Fifth Floor Boston, MA 02110-1301 USA http://www.fsf.org/about/contact/ Regards Matej Mužila

1 0

[Maria-developers] master.info and relay-log.info location
by Tim Callaghan 15 Sep '14

15 Sep '14

Has any work been done in MariaDB 10.0 or 10.1 to move them from their file-based location to within XtraDB (and thus potentially TokuDB)? The knowledge base article at https://mariadb.com/kb/en/mariadb/development/general-development-informati… lists the following as a roadmap feature: - Transactional storage of slave state, rather than file-based master.info and relay-log.info . So the slave can recover consistently after a crash.

2 1

[Maria-developers] contribution procedure
by Jonas Oreland 12 Sep '14

12 Sep '14

Hi Serg, hope all is well. I created https://mariadb.atlassian.net/browse/MDEV-6712 for a plugin feature I wrote, and now I humbly ponder on how I get this review/triaged/something. What would be your best advice? /Jonas

2 1

[Maria-developers] How to get all fields in a given index?
by Rongrong 10 Sep '14

10 Sep '14

Hi, I'm trying to implement some feature that requires to get the list of fields for a given index. I'm trying to follow the optimizer code to find some clue but am not successful so far. Can anyone give me some pointers? Thanks! Rongrong

2 2

[Maria-developers] Total breakdown of development procedures :-(
by Kristian Nielsen 10 Sep '14

10 Sep '14

Today, I learned that a large change (the galera merge) has been pushed into 10.1 on August 27. At that point, there was not a _single_ builder in buildbot that was green! As a result, we now have an extremely serious regression in the server caused by this push, easily seen in the testsuite. This is a disaster! Someone (or more likely everyone) have forgotten the basic rules, rules that have been in place since the beginning of MariaDB, and which are essential to ensuring the necessary quality of a product used in production-critical environments every day: - No pushes into a tree with known test failures in Buildbot. - No pushes of larger changes without first seeing them green in Buildbot in a feature tree. - No pushes without review. - No pushes of new features that are not of beta quality. Must be feature complete, must have passed tests and review, must have documentation ready. I have asked Jan to revert the Galera merge until it can be tested green in Buildbot and passed RQG replication tests by Elena. But the problem here is clearly larger than any single patch. Things have degenerated far beyond what is even remotely acceptable. We need to step back and start following the rules to the letter to recover from this. - Kristian.

1 0

Re: [Maria-developers] [Commits] Rev 4385: MDEV-6459 max_relay_log_size and sql_slave_skip_counter misbehave on PPC64 in lp:~maria-captains/maria/10.0
by Sergey Vojtovich 09 Sep '14

09 Sep '14

Hi Sergei, just curious: any special reason to make these variables ulonglong? Both seem to fit into uint. Thanks, Sergey On Tue, Sep 09, 2014 at 04:52:12PM +0200, Sergei Golubchik wrote: > At lp:~maria-captains/maria/10.0 > > ------------------------------------------------------------ > revno: 4385 > revision-id: sergii(a)pisem.net-20140909145212-62txhaq57k301s16 > parent: sergii(a)pisem.net-20140909130235-h2sy95zwu55701l0 > fixes bug: https://mariadb.atlassian.net/browse/MDEV-6459 > committer: Sergei Golubchik <sergii(a)pisem.net> > branch nick: 10.0 > timestamp: Tue 2014-09-09 16:52:12 +0200 > message: > MDEV-6459 max_relay_log_size and sql_slave_skip_counter misbehave on PPC64 > > make slave_skip_counter and max_relay_log_size ulonglong > (sysvars should generally never be ulong) > === modified file 'sql/rpl_rli.h' > --- a/sql/rpl_rli.h 2014-09-08 10:59:57 +0000 > +++ b/sql/rpl_rli.h 2014-09-09 14:52:12 +0000 > @@ -251,10 +251,11 @@ class Relay_log_info : public Slave_repo > errors, and have been manually applied by DBA already. > Must be ulong as it's refered to from set_var.cc > */ > - volatile ulong slave_skip_counter; > + volatile ulonglong slave_skip_counter; > + ulonglong max_relay_log_size; > + > volatile ulong abort_pos_wait; /* Incremented on change master */ > volatile ulong slave_run_id; /* Incremented on slave start */ > - ulong max_relay_log_size; > mysql_mutex_t log_space_lock; > mysql_cond_t log_space_cond; > /* > > === modified file 'sql/sql_class.h' > --- a/sql/sql_class.h 2014-07-31 09:08:56 +0000 > +++ b/sql/sql_class.h 2014-09-09 14:52:12 +0000 > @@ -519,6 +519,14 @@ typedef struct system_variables > ulonglong sortbuff_size; > ulonglong group_concat_max_len; > ulonglong default_regex_flags; > + > + /** > + Place holders to store Multi-source variables in sys_var.cc during > + update and show of variables. > + */ > + ulonglong slave_skip_counter; > + ulonglong max_relay_log_size; > + > ha_rows select_limit; > ha_rows max_join_size; > ha_rows expensive_subquery_limit; > @@ -587,12 +595,6 @@ typedef struct system_variables > */ > uint32 gtid_domain_id; > uint64 gtid_seq_no; > - /** > - Place holders to store Multi-source variables in sys_var.cc during > - update and show of variables. > - */ > - ulong slave_skip_counter; > - ulong max_relay_log_size; > > /** > Default transaction access mode. READ ONLY (true) or READ WRITE (false). > > === modified file 'sql/sys_vars.cc' > --- a/sql/sys_vars.cc 2014-08-07 16:06:56 +0000 > +++ b/sql/sys_vars.cc 2014-09-09 14:52:12 +0000 > @@ -4248,11 +4248,11 @@ static Sys_var_uint Sys_slave_net_timeou > Return 0 + warning if it doesn't exist > */ > > -ulong Sys_var_multi_source_ulong:: > -get_master_info_ulong_value(THD *thd, ptrdiff_t offset) > +ulonglong Sys_var_multi_source_ulonglong:: > +get_master_info_ulonglong_value(THD *thd, ptrdiff_t offset) > { > Master_info *mi; > - ulong res= 0; // Default value > + ulonglong res= 0; // Default value > mysql_mutex_unlock(&LOCK_global_system_variables); > mysql_mutex_lock(&LOCK_active_mi); > mi= master_info_index-> > @@ -4261,7 +4261,7 @@ get_master_info_ulong_value(THD *thd, pt > if (mi) > { > mysql_mutex_lock(&mi->rli.data_lock); > - res= *((ulong*) (((uchar*) mi) + master_info_offset)); > + res= *((ulonglong*) (((uchar*) mi) + master_info_offset)); > mysql_mutex_unlock(&mi->rli.data_lock); > } > mysql_mutex_unlock(&LOCK_active_mi); > @@ -4273,7 +4273,7 @@ get_master_info_ulong_value(THD *thd, pt > bool update_multi_source_variable(sys_var *self_var, THD *thd, > enum_var_type type) > { > - Sys_var_multi_source_ulong *self= (Sys_var_multi_source_ulong*) self_var; > + Sys_var_multi_source_ulonglong *self= (Sys_var_multi_source_ulonglong*) self_var; > bool result= true; > Master_info *mi; > > @@ -4310,16 +4310,12 @@ static bool update_slave_skip_counter(sy > return false; > } > > - > -static Sys_var_multi_source_ulong > -Sys_slave_skip_counter("sql_slave_skip_counter", > - "Skip the next N events from the master log", > - SESSION_VAR(slave_skip_counter), > - NO_CMD_LINE, > - my_offsetof(Master_info, rli.slave_skip_counter), > - VALID_RANGE(0, UINT_MAX), DEFAULT(0), BLOCK_SIZE(1), > - ON_UPDATE(update_slave_skip_counter)); > - > +static Sys_var_multi_source_ulonglong Sys_slave_skip_counter( > + "sql_slave_skip_counter", "Skip the next N events from the master log", > + SESSION_VAR(slave_skip_counter), NO_CMD_LINE, > + MASTER_INFO_VAR(rli.slave_skip_counter), > + VALID_RANGE(0, UINT_MAX), DEFAULT(0), BLOCK_SIZE(1), > + ON_UPDATE(update_slave_skip_counter)); > > static bool update_max_relay_log_size(sys_var *self, THD *thd, Master_info *mi) > { > @@ -4328,17 +4324,14 @@ static bool update_max_relay_log_size(sy > return false; > } > > -static Sys_var_multi_source_ulong > -Sys_max_relay_log_size( "max_relay_log_size", > - "relay log will be rotated automatically when the " > - "size exceeds this value. If 0 are startup, it's " > - "set to max_binlog_size", > - SESSION_VAR(max_relay_log_size), > - CMD_LINE(REQUIRED_ARG), > - my_offsetof(Master_info, rli.max_relay_log_size), > - VALID_RANGE(0, 1024L*1024*1024), DEFAULT(0), > - BLOCK_SIZE(IO_SIZE), > - ON_UPDATE(update_max_relay_log_size)); > +static Sys_var_multi_source_ulonglong Sys_max_relay_log_size( > + "max_relay_log_size", > + "relay log will be rotated automatically when the size exceeds this " > + "value. If 0 are startup, it's set to max_binlog_size", > + SESSION_VAR(max_relay_log_size), CMD_LINE(REQUIRED_ARG), > + MASTER_INFO_VAR(rli.max_relay_log_size), > + VALID_RANGE(0, 1024L*1024*1024), DEFAULT(0), BLOCK_SIZE(IO_SIZE), > + ON_UPDATE(update_max_relay_log_size)); > > static Sys_var_charptr Sys_slave_skip_errors( > "slave_skip_errors", "Tells the slave thread to continue " > > === modified file 'sql/sys_vars.h' > --- a/sql/sys_vars.h 2014-07-18 15:45:21 +0000 > +++ b/sql/sys_vars.h 2014-09-09 14:52:12 +0000 > @@ -1993,7 +1993,8 @@ class Sys_var_replicate_events_marked_fo > like sql_slave_skip_counter are GLOBAL. > */ > > -class Sys_var_multi_source_ulong; > +#define MASTER_INFO_VAR(X) my_offsetof(Master_info, X), sizeof(Master_info::X) > +class Sys_var_multi_source_ulonglong; > class Master_info; > > typedef bool (*on_multi_source_update_function)(sys_var *self, THD *thd, > @@ -2002,31 +2003,27 @@ bool update_multi_source_variable(sys_va > THD *thd, enum_var_type type); > > > -class Sys_var_multi_source_ulong :public Sys_var_ulong > +class Sys_var_multi_source_ulonglong :public Sys_var_ulonglong > { > ptrdiff_t master_info_offset; > on_multi_source_update_function update_multi_source_variable_func; > public: > - Sys_var_multi_source_ulong(const char *name_arg, > + Sys_var_multi_source_ulonglong(const char *name_arg, > const char *comment, int flag_args, > ptrdiff_t off, size_t size, > CMD_LINE getopt, > ptrdiff_t master_info_offset_arg, > - ulong min_val, ulong max_val, ulong def_val, > - uint block_size, > + size_t master_info_arg_size, > + ulonglong min_val, ulonglong max_val, > + ulonglong def_val, uint block_size, > on_multi_source_update_function on_update_func) > - :Sys_var_ulong(name_arg, comment, flag_args, off, size, > - getopt, min_val, max_val, def_val, block_size, > - 0, VARIABLE_NOT_IN_BINLOG, 0, update_multi_source_variable), > + :Sys_var_ulonglong(name_arg, comment, flag_args, off, size, > + getopt, min_val, max_val, def_val, block_size, > + 0, VARIABLE_NOT_IN_BINLOG, 0, update_multi_source_variable), > master_info_offset(master_info_offset_arg), > update_multi_source_variable_func(on_update_func) > { > - } > - bool session_update(THD *thd, set_var *var) > - { > - session_var(thd, ulong)= (ulong) (var->save_result.ulonglong_value); > - /* Value should be moved to multi_master in on_update_func */ > - return false; > + SYSVAR_ASSERT(master_info_arg_size == size); > } > bool global_update(THD *thd, set_var *var) > { > @@ -2039,9 +2036,9 @@ class Sys_var_multi_source_ulong :public > } > uchar *session_value_ptr(THD *thd,LEX_STRING *base) > { > - ulong *tmp, res; > - tmp= (ulong*) (((uchar*)&(thd->variables)) + offset); > - res= get_master_info_ulong_value(thd, master_info_offset); > + ulonglong *tmp, res; > + tmp= (ulonglong*) (((uchar*)&(thd->variables)) + offset); > + res= get_master_info_ulonglong_value(thd, master_info_offset); > *tmp= res; > return (uchar*) tmp; > } > @@ -2049,7 +2046,7 @@ class Sys_var_multi_source_ulong :public > { > return session_value_ptr(thd, base); > } > - ulong get_master_info_ulong_value(THD *thd, ptrdiff_t offset); > + ulonglong get_master_info_ulonglong_value(THD *thd, ptrdiff_t offset); > bool update_variable(THD *thd, Master_info *mi) > { > return update_multi_source_variable_func(this, thd, mi); > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

2 1

[Maria-developers] Patch: CREATE VIEW test program
by James Briggs 08 Sep '14

08 Sep '14

Hi List. Below is tests/view.pl for testing CREATE VIEW options. It tests several thousand permutations and was used to find MDEV-6709 Thanks, James Briggs. -- Available for MySQL/NoSQL DBA/Programming in San Jose area or remote. tests/view.pl: #!/usr/bin/perl # Program: view.pl # Licence: GPL version 2 # Author: James Briggs # Date: 2014 09 07 # Env: Perl 5 # Note: requires CREATE [[NO] FORCE] option # # Testing of views. # use strict; use warnings; use DBI; use Getopt::Long; $|=1; use vars qw($opt_help $opt_Information $opt_force $opt_debug $opt_verbose $opt_root_user $opt_root_password $opt_user $opt_password $opt_database $opt_host $opt_silent); $opt_help = $opt_Information = $opt_force = $opt_debug = $opt_verbose = $opt_silent = 0; $opt_host = "localhost", $opt_root_user = "root"; $opt_root_password = ""; $opt_user = "view_user"; $opt_password = "view_user"; $opt_database = "view_test"; $opt_force = 1; my $version = "1.0"; my $opt_table="table1"; my $opt_user2 = $opt_user . '2'; GetOptions("Information","help","server=s","root-user=s","root-password=s","user","password=s","database=s","force","host=s","debug","verbose","silent") || usage(); usage() if ($opt_help || $opt_Information); # magic constants use constant N_TEST_RECORDS => 3; use constant HAS_VIEW_GRANTS => 0; use constant NO_VIEW_GRANTS => 1; my @cols = qw[id col1 col2 col3 col4]; my $tmp_table="/tmp/mysql-view.test"; unlink($tmp_table); # %roles array offets use constant U_DBH => 0; use constant U_PRIV_LEVEL => 1; use constant U_USER => 2; use constant U_HOST => 3; use constant U_PASSWORD => 4; # run tests with multiple user accounts from SUPER to very basic privs my %roles = ( # [ U_DBH, U_PRIV_LEVEL, U_USER, U_HOST, U_PASSWORD ] root => [ undef, HAS_VIEW_GRANTS, $opt_root_user, $opt_host, $opt_root_password ], power => [ undef, HAS_VIEW_GRANTS, $opt_user, $opt_host, $opt_password ], crud => [ undef, NO_VIEW_GRANTS, $opt_user2, $opt_host, $opt_password], # use the non-root user account again, this time with less privs ); if (!$opt_force) { print_info(); } # # setup test database # my $dbh = user_connect($opt_root_user,$opt_root_password, 0, 'test'); # $opt_database may not exist yet, so connect with 'test' $roles{'root'}->[U_DBH] = $dbh; test_query('root', "drop database if exists $opt_database"); # drop database to quickly drop any tables and views test_query('root', "create database $opt_database"); test_query('root', "use $opt_database"); # # setup test table # test_query('root', "create table $opt_table (id int primary key auto_increment, col1 int, col2 int, col3 int, col4 int)"); my @c = @cols; # column names for test database shift @c; # remove first column (id) my $cols = join ',', @c; # squash array into a string for my $i (1..N_TEST_RECORDS) { test_query('root', "insert into $opt_table ($cols) values (2, 3, 4, 5)"); } test_query('root', "grant select, insert, update, delete, create, drop, create view, show view on $opt_database.* to '$opt_user'\@'$opt_host' identified by '$opt_password'"); my $dbh_power = user_connect($opt_user, $opt_password, 0); test_query('root', "grant select, insert, update, delete on $opt_database.* to '$opt_user2'\@'$opt_host' identified by '$opt_password'"); my $dbh_crud = user_connect($opt_user2, $opt_password, 0); $roles{'power'}->[0] = $dbh_power; $roles{'crud'}->[0] = $dbh_crud; # # test views # # query array offsets use constant Q_QRY => 0; use constant Q_HI_PRIV => 1; use constant Q_LO_PRIV => 2; use constant Q_OUTPUT => 3; use constant Q_COMMENT => 4; my @t0 = ( # [ query, ignore_failure_hi_priv, ignore_failure_lo_priv, result, comment ] # [ Q_QRY, Q_HI_PRIV, Q_LO_PRIV, Q_OUTPUT, Q_COMMENT ] [ "create view $opt_table as select * from $opt_table", 1, 1, undef, "should fail - duplicate object name" ], [ "create view view1 as select * from ${opt_table}2", 1, 1, undef, "should fail - no base table found" ], [ "create view view1 as select * from $opt_table", 0, 1, undef, "" ], [ "select count(*) from view1", 0, 1, N_TEST_RECORDS, "" ], [ "create definer = current_user() sql security invoker view v1 as select 1", 0, 1, undef, "" ], [ "drop view view1", 0, 1, undef, "" ], [ "drop view v1", 0, 1, undef, "" ], ); test_driver('original create view commands', \%roles, \@t0); # # test new CREATE NO FORCE VIEW view options in 10.1.x # # CREATE FORCE VIEW should work the same as default (omitted) my @t1 = ( [ "create no force view $opt_table as select * from $opt_table", 1, 1, undef, "should fail - duplicate object name" ], [ "create no force view view1 as select * from $opt_table", 0, 1, undef, "" ], [ "create no force view view1 as select * from $opt_table", 1, 1, undef, "should fail - duplicate object name" ], [ "select count(*) from view1", 0, 1, N_TEST_RECORDS, "" ], [ "drop view view1", 0, 1, undef, "" ], ); test_driver('new CREATE NO FORCE VIEW options in 10.1.x', \%roles, \@t1); # # test new CREATE FORCE VIEW options in 10.1.x # # Similar to Oracle Enterprise, behavior of CREATE FORCE VIEW: # # - no base table needs exist at creation time # - thus no table or column access rights need exist # # - however, CREATE VIEW and SHOW VIEW should be enforced my @t2 = ( [ "create force view $opt_table as select id, col1 from $opt_table", 1, 1, undef, "should fail - duplicate object name" ], [ "create force view view1 as select id, col1 from $opt_table", 0, 1, undef, "" ], [ "create force view view1 as select id, col1 from $opt_table", 1, 1, undef, "should fail - duplicate object name" ], [ "create force view view2 as select id, col1 from ${opt_table}2", 0, 1, undef, "" ], # failed on "CREATE ALGORITHM=UNDEFINED DEFINER=`root`@`localhost` SQL SECURITY DEFINER VIEW `view1` AS select `*` AS `*` from `table1`" # note two AS symbols # [ "select count(*) from view1", 0, 1, N_TEST_RECORDS, "" ], [ "drop view view1, view2", 0, 1, undef, "" ], ); test_driver('new CREATE VIEW FORCE view options in 10.1.x', \%roles, \@t2); # test views by automatically generating thousands of permutation of CREATE VIEW gen_permutations(); # # Clean up # unlink($tmp_table); test_query('root', "revoke all privileges, grant option from '$opt_user'\@'%'"); test_query('root', "revoke all privileges, grant option from '$opt_user2'\@'%'"); test_query('root', "drop database if exists $opt_database"); print "end of test\n"; exit 0; # # do permuted tests # # Todo: # # - derived tables # - I_S tables # - views of views # - invalid syntax sub gen_permutations { # CREATE # [OR REPLACE] # [ALGORITHM = {UNDEFINED | MERGE | TEMPTABLE}] # [DEFINER = { user | CURRENT_USER }] # [SQL SECURITY { DEFINER | INVOKER }] # [[NO] FORCE] # VIEW view_name [(column_list)] # AS select_statement # [WITH [CASCADED | LOCAL] CHECK OPTION] use constant P_CREATE => 0; use constant P_ALGORITHM => 1; use constant P_DEFINER => 2; use constant P_SQL_SECURITY => 3; use constant P_FORCE => 4; use constant P_WITH => 5; # insert a blank string to make the option skippable my @p = ( [ 'create', 'create or replace'], [ '', 'undefined', 'merge', 'temptable' ], [ '', 'user', 'current_user'], [ '', 'definer', 'invoker' ], [ '', 'force', 'no force'], [ '', 'cascaded', 'local'], ); my $permutations = 0; my $sql = ''; my $quiet = 1; print "Trying create view permutations\n\n"; for my $role (sort { $b cmp $a } keys %roles) { for my $create (@{$p[P_CREATE]}) { for my $algorithm (@{$p[P_ALGORITHM]}) { for my $definer (@{$p[P_DEFINER]}) { for my $sql_security (@{$p[P_SQL_SECURITY]}) { for my $force (@{$p[P_FORCE]}) { for my $col_list (('', $cols)) { for my $columns (('*', $cols)) { for my $with (@{$p[P_WITH]}) { next if ($columns eq '*' && $force eq 'force'); # ambiguous view definition $sql = $create . ' '; $sql .= "ALGORITHM = $algorithm " if $algorithm ne ''; if ($definer ne '') { if ($definer eq 'user') { $sql .= "DEFINER = '$roles{$role}->[U_USER]'\@'$roles{$role}->[U_HOST]' "; } else { $sql .= "DEFINER = $definer "; } } $sql .= "$force " if $force ne ''; $sql .= "VIEW "; $sql .= "view1 "; $sql .= "($col_list) " if $col_list ne '' and $col_list eq $columns; # if you use $col_list, the number of columns must match $columns $sql .= "AS SELECT $columns FROM $opt_table "; $sql .= "WITH $with CHECK OPTION" if $with ne ''; print "$permutations:$role: $sql\n"; my $will_error = 0; $will_error = 1 if ($role eq 'crud') # no grants || ($algorithm eq 'temptable' and $with ne '') # always an error to CHECK a TEMPTABLE # || ($definer eq 'user' and $role ne 'root') # super priv needed for DEFINER user ; test_query('root', 'drop view if exists view1', 0, $quiet); test_query($role, $sql, $will_error, $quiet); test_query($role, 'select count(*) from view1', $will_error, $quiet); if (!$will_error && $roles{$role}->[U_PRIV_LEVEL] == HAS_VIEW_GRANTS) { if (db_cmp_count($roles{'root'}->[U_DBH], 'select count(*) from view1', N_TEST_RECORDS )) { die "error: wrong row count for '$sql'"; } } $permutations++; } } } } } } } } } print "total permutations = $permutations\n"; return 0; } sub test_driver { my ($heading, $r_roles, $r_qry) = @_; print $heading, "\n\n"; for my $role (sort { $b cmp $a } keys %$r_roles) { test_query('root', "drop view if exists view1"); for my $q (@$r_qry) { my $flag_fail = ($r_roles->{$role}->[U_PRIV_LEVEL] == HAS_VIEW_GRANTS) ? $q->[Q_HI_PRIV] : $q->[Q_LO_PRIV]; print "$role '$q->[Q_QRY]'", ($flag_fail ? ' should fail' : ' should pass'), "\n"; test_query($role, $q->[Q_QRY], $flag_fail, 0); if (defined $q->[Q_OUTPUT]) { my $ret = db_cmp_count($r_roles->{'root'}->[U_DBH], $q->[Q_QRY], ($flag_fail ? undef : $q->[Q_OUTPUT])); die if $ret; } } } print "\n\n"; } sub usage { print <<EOF; $0 Ver $version This program tests that the VIEW commands works by creating a temporary database ($opt_database) and users ($opt_user, $opt_user2). Options: --database (Default $opt_database) In which database the test tables are created. --force Don''t ask any question before starting this test. --host='host name' (Default $opt_host) Host name where the database server is located. --Information --help Print this help --root-password Password for root-user. --user (Default $opt_user) A non-existing user on which we will test view commands --password Password for non-root-user. --verbose Write all queries when we are execute them. --root-user='user name' (Default $opt_root_user) superuser for creating tables and grants EOF exit(0); } sub print_info { my $tmp; print <<EOF; This test will do view statements against the $opt_database database ! the $opt_database database and $opt_user user will be created and deleted ! EOF while (1) { print "Start test (yes/no) ? "; $tmp=<STDIN>; chomp($tmp); $tmp=lc($tmp); last if ($tmp =~ /^yes$/i); exit 1 if ($tmp =~ /^n/i); print "\n"; } } sub user_connect { my ($user, $password, $ignore_error, $db) = @_; $db = $opt_database if not defined $db or $db eq ''; print "Connecting $user\n" if ($opt_verbose); my $dbh =DBI->connect("DBI:mysql:$db:$opt_host",$user, $password, { PrintError => 0}); if (!$dbh) { if ($opt_verbose || !$ignore_error) { print "error on connect: $DBI::errstr\n"; } if (!$ignore_error) { die "The above should not have failed!"; } } elsif ($ignore_error) { die "Connect succeeded when it shouldn't have !\n"; } else { return $dbh; } } sub test_query { my ($role, $query, $ignore_error, $quiet) = @_; my ($package, $filename, $line) = caller; my $dbh = $roles{$role}->[U_DBH]; if (defined $dbh && !$dbh->ping) { $roles{$role}->[U_DBH] = user_connect($roles{$role}->[U_USER], $roles{$role}->[U_PASSWORD], $ignore_error); $dbh = $roles{$role}->[U_DBH]; } elsif (not defined $dbh) { $roles{$role}->[U_DBH] = user_connect($roles{$role}->[U_USER], $roles{$role}->[U_PASSWORD], $ignore_error); $dbh = $roles{$role}->[U_DBH]; } if (do_query($dbh, $query, $ignore_error, $quiet)) { if (!defined($ignore_error)) { print "error:$line: This query should not have failed: '$query', do SHOW CREATE VIEW VIEW_TEST.VIEW1 to troubleshoot.\n"; exit 1; } } elsif (defined($ignore_error) && $ignore_error == 1) { print "error:$line: This query should not have succeeded: '$query'\n"; exit 1; } } sub do_query { my ($my_dbh, $query, $ignore_error, $quiet) = @_; my ($sth, $row, $fatal_error); print "$query\n" if ($opt_debug || $opt_verbose); if (!($sth= $my_dbh->prepare($query))) { print "error in prepare: $DBI::errstr\n"; return 1; } if (!$sth->execute) { $fatal_error= ($DBI::errstr =~ /parse error/); if (!$ignore_error || ($opt_verbose && $ignore_error != 3) || $fatal_error) { print "error in execute: $DBI::errstr\n"; { my $cmd = "show create view $opt_database.view1"; my $out = `mysql -h $opt_host -u root -p$opt_root_password -e '$cmd'`; print "$cmd: $out\n"; print '"*** You have the SQL parser bug: AS select `*` AS `*` ***' . "\n" if $out =~ / AS SELECT .* AS /i; } } die if ($fatal_error); $sth->finish; return 1; } if (!$opt_silent and !$quiet) { my $found = 0; while (($row=$sth->fetchrow_arrayref)) { $found = 1; my $tab = ''; for my $col (@$row) { print $tab; print defined($col) ? $col : "NULL"; $tab="\t"; } print "\n"; } print "\n" if $found; } $sth->finish; return 0; } # Note: cmp_tmp_table is not currently used, but available for future use again sub cmp_tmp_table { my ($s) = @_; if (not defined $s) { if (-e $tmp_table) { return 1; } else { return 0; } } $s =~ s/\n+$//g; # remove trailing blanks open X, "<", $tmp_table or return 2; local($/)=''; my $t = <X>; close X; unlink($tmp_table) or warn "debug: cannot unlink tmp table"; $t =~ s/\n+$//g; if ($s ne $t) { return 1; } return 0; } sub db_cmp_count { my ($dbh, $q, $s) = @_; my $sth = $dbh->prepare($q); my $ret = $sth->execute() || do { return 0 if not defined $s; return 2; }; my $out = ''; if (my (@row) = $sth->fetchrow_array()) { $out = $row[0]; } $sth->finish; return 1 if $out != $s; return 0; } # The End.

1 0

[Maria-developers] mariadb native client -- how to call RPMs?
by Honza Horak 05 Sep '14

05 Sep '14

Hi, in Fedora, we're about to package mariadb-native-client, which we'd like to utilize in Fedora as much as possible since using this piece of software we can break the dependency on whole mariadb server package and thus make the base system more compact. However, since mariadb.org does not provide RPMs yet, I'm wondering what would be the best name for such RPM package. The tar ball is called mariadb_client.tar.gz but the project is called mariadb-native-client on launchpad, which I like more btw. because it differs more from mariadb-client package (which is actually a subpackage of mariadb). So, is mariadb-native-client a good choice or is there a better one? Cheers, Honza

3 3

[Maria-developers] no information about references - mysql bug 73757
by Roberto Spadim 04 Sep '14

04 Sep '14

Hi guys, there's a bug report about no precise information to solve a problem: http://bugs.mysql.com/bug.php?id=73757 test case: create table t11 (f1 integer primary key) engine innodb; alter table t11 add constraint c1 foreign key (f1) references t1(f1); /* Erro SQL (1005): Can't create table `teste`.`#sql-3f90_25c30` (errno: 150 "Foreign key constraint is incorrectly formed") */ should be better explain that reference table t1 don't exists or something like it should i report at jira, or just mysql bug database is ok? -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

2 2

[Maria-developers] Updating MySQL Client Protocol: Possible and still be backward compatible?
by Adam Scott 04 Sep '14

04 Sep '14

Is it possible to have mysql client protocol return the OS username and allow the protocol to backward compatible? Looking to see if reasonable to provide same functionality as Oracle does with select sys_context( 'userenv', 'os_user' ) from dual;

2 1

[Maria-developers] doubt about mdev 4682 - tests
by Roberto Spadim 04 Sep '14

04 Sep '14

Guys, i'm with time and was looking an old patch about query cache information schema https://mariadb.atlassian.net/browse/MDEV-4682 it add an statistic about query cache hits my doubt is... i was trying to understand how could i create tests, since the result of statistic isn't deterministic how should i execute a test case like this? columns that i have doubt: columns ... QUERY_ROWS <- deterministic SELECT_ROWS_READ <- deterministic QUERY_HITS <- deterministic QUERY_HITS_PERIOD_LOW <- non deterministic QUERY_HITS_PERIOD_HIGH <- non deterministic QUERY_HITS_PERIOD_OUTLIERS <- non deterministic QUERY_HITS_TOTAL_TIME_US <- non deterministic QUERY_HITS_MEAN_PERIOD_US <- non deterministic QUERY_HITS_MEAN_PERIOD_LOW_US <- non deterministic QUERY_HITS_MEAN_PERIOD_HIGH_US <- non deterministic QUERY_INSERT_TIME <- probably deterministic (or not?) QUERY_LAST_HIT_TIME <- probably deterministic (or not?) SELECT_EXPEND_TIME_US <- non deterministic SELECT_LOCK_TIME_US <- non deterministic ... -- Roberto Spadim

2 1

[Maria-developers] Query in size
by Kavneet Kaur 04 Sep '14

04 Sep '14

Hi all, I just when through a revision~maria-captains/mariadb-java-client/trunk : /src/test/java/org/mariadb/jdbc/BigQueryTest.java (revision 516)Big Query activated in this code contains a character array of size 20000000. This is something that can be modified. All big queries possible cannot occupy this space. This is sort of leading to more of wastage of memory. Can anyone help me understand why this size has been selected when the same could be done at a size (50-90)% lesser. Regards Kavneet Kaur

2 1

Re: [Maria-developers] Interest in contributing to MariaDB
by Djimeli Konrad 03 Sep '14

03 Sep '14

Thanks for the reply Roberto, i was beginning to get confused because the code i got from github was on c/c++. On 3 September 2014 21:13, Djimeli Konrad <djkonro35(a)gmail.com> wrote: > Thanks for the reply Roberto, i was beginning to get confused because the > code i got from github was on c/c++. > > > On 3 September 2014 20:39, Roberto Spadim <roberto(a)spadim.com.br> wrote: > >> ???? java? no guy, mariadb database source code is C/C++ check : >> >> github : https://github.com/MariaDB/server >> and issue tracker: https://mariadb.atlassian.net >> there's mariadb knowledge: https://mariadb.com/kb/en/ >> there're some book about mariadb/mysql internall and mysql internall >> documentation: http://dev.mysql.com/doc/internals/en/ >> >> java is a connect, you can check more at mariadb page: >> https://mariadb.com/products/connectors-plugins >> >> >> 2014-09-03 17:15 GMT-03:00 Kavneet Kaur <kavneetk(a)outlook.com>: >> >> Hi Djimeli, >>> >>> As far as I have studied about the code it's in java. If you wish to >>> begin it from basic of database, you can go for installation. You should go >>> through documentations also. Explore your interest and surf as much as you >>> can. >>> >>> All the best! >>> >>> > Date: Wed, 3 Sep 2014 15:35:24 +0000 >>> > From: djkonro35(a)gmail.com >>> > To: maria-developers(a)lists.launchpad.net >>> > Subject: [Maria-developers] Interest in contributing to MariaDB >>> >>> > >>> > Hello >>> > >>> > My name is Djimeli Konrad and i am primarily a c/c++ >>> > programmer with some experience in php and mysql. I am new to open >>> > source and would like to contribute to mariadb so i wish i could get >>> > some pointers on how to begin. >>> > >>> > Thanks >>> > konrad >>> > >>> > _______________________________________________ >>> > Mailing list: https://launchpad.net/~maria-developers >>> > Post to : maria-developers(a)lists.launchpad.net >>> > Unsubscribe : https://launchpad.net/~maria-developers >>> > More help : https://help.launchpad.net/ListHelp >>> >>> _______________________________________________ >>> Mailing list: https://launchpad.net/~maria-developers >>> Post to : maria-developers(a)lists.launchpad.net >>> Unsubscribe : https://launchpad.net/~maria-developers >>> More help : https://help.launchpad.net/ListHelp >>> >>> >> >> >> -- >> Roberto Spadim >> SPAEmpresarial >> Eng. Automação e Controle >> > >

1 0

[Maria-developers] Interest in contributing to MariaDB
by Djimeli Konrad 03 Sep '14

03 Sep '14

Hello My name is Djimeli Konrad and i am primarily a c/c++ programmer with some experience in php and mysql. I am new to open source and would like to contribute to mariadb so i wish i could get some pointers on how to begin. Thanks konrad

3 2

[Maria-developers] New MariaDB builds: regular trusty and special kvm-deb-sid-pbuilder-amd and (Re: MariaDB)
by Otto Kekäläinen 01 Sep '14

01 Sep '14

Hello! 2014-02-04 20:41 GMT+02:00 Daniel Bartholomew <dbart(a)mariadb.com>: > On Tue, Feb 4, 2014 at 5:39 AM, Kristian Nielsen > <knielsen(a)knielsen-hq.org> wrote: >>> And while you are at it, please also consider setting up an instance >>> 'kvm-deb-sid-pbuilder-amd64' that provides a Debian unstable >> >> Daniel, can you prepare new sets of VMs for Ubuntu trusty and Debian sid? >> Then I can try to setup Buildbot to do these builds. > > Ok, will do. I should have them up and running before the end of the week. Any updates on this?

3 22

Re: [Maria-developers] [Commits] eecac19: MDEV-6513 deprecate engine_condition_pushdown value of the @@optimizer_switch
by Oleksandr Byelkin 28 Aug '14

28 Aug '14

Hi! Everything is OK. Except one small note in two places. But it is OK to push as is. 26.08.2014 17:17, serg(a)mariadb.org пишет: > revision-id: eecac1976469d513062a0741005afc357b897871 > parent(s): f1c1c04a34f39bcd6622b9886fb089dd41e51103 > committer: Sergei Golubchik > branch nick: maria > timestamp: 2014-08-25 23:13:37 +0200 > message: > > MDEV-6513 deprecate engine_condition_pushdown value of the @@optimizer_switch > > * ignore the OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN bit > * issue a deprecation warning on 'engine_condition_pushdown=on' > * remove unused remains of the old pre-5.5 engine_condition_pushdown variable > > --- [skip] > --- a/sql/handler.h > +++ b/sql/handler.h > @@ -197,17 +197,11 @@ enum enum_alter_inplace_result { > #define HA_RECORD_MUST_BE_CLEAN_ON_WRITE (1ULL << 41) > > /* > - Table condition pushdown must be performed regardless of > - 'engine_condition_pushdown' setting. > - > - This flag is aimed at storage engines that come with "special" predicates > - that can only be evaluated inside the storage engine. > - For example, when one does > - select * from sphinx_table where query='{fulltext_query}' > - then the "query=..." condition must be always pushed down into storage > - engine. > + This storage engine supports condition pushdown > */ > -#define HA_MUST_USE_TABLE_CONDITION_PUSHDOWN (1ULL << 42) > +#define HA_CAN_TABLE_CONDITION_PUSHDOWN (1ULL << 42) > +/* old name for the same flag */ > +#define HA_MUST_USE_TABLE_CONDITION_PUSHDOWN HA_CAN_TABLE_CONDITION_PUSHDOWN > > /** > The handler supports read before write removal optimization > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index 57d2cc6..fe52a85 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -8574,18 +8574,6 @@ mysqld_get_one_option(int optid, > } > break; > #endif /* defined(ENABLED_DEBUG_SYNC) */ > - case OPT_ENGINE_CONDITION_PUSHDOWN: > - /* > - The last of --engine-condition-pushdown and --optimizer_switch on > - command line wins (see get_options(). > - */ > - if (global_system_variables.engine_condition_pushdown) > - global_system_variables.optimizer_switch|= > - OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN; > - else > - global_system_variables.optimizer_switch&= > - ~OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN; > - break; > case OPT_LOG_ERROR: > /* > "No --log-error" == "write errors to stderr", > @@ -8964,10 +8952,6 @@ static int get_options(int *argc_ptr, char ***argv_ptr) > &extra_connection_count); > #endif > > - global_system_variables.engine_condition_pushdown= > - MY_TEST(global_system_variables.optimizer_switch & > - OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN); > - > opt_readonly= read_only; > > /* > diff --git a/sql/mysqld.h b/sql/mysqld.h > index 21c020b..7a0a75c 100644 > --- a/sql/mysqld.h > +++ b/sql/mysqld.h > @@ -554,7 +554,6 @@ enum options_mysqld > OPT_DEBUG_SYNC_TIMEOUT, > OPT_DELAY_KEY_WRITE_ALL, > OPT_DEPRECATED_OPTION, > - OPT_ENGINE_CONDITION_PUSHDOWN, > OPT_IGNORE_DB_DIRECTORY, > OPT_ISAM_LOG, > OPT_KEY_BUFFER_SIZE, > diff --git a/sql/records.cc b/sql/records.cc > index 1b230c4..cbcdb36 100644 > --- a/sql/records.cc > +++ b/sql/records.cc > @@ -287,8 +287,8 @@ bool init_read_record(READ_RECORD *info,THD *thd, TABLE *table, > thd->variables.read_buff_size); > } > /* Condition pushdown to storage engine */ > - if ((thd->variables.optimizer_switch & > - OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN) && > + if ((table->file->ha_table_flags() & > + HA_MUST_USE_TABLE_CONDITION_PUSHDOWN) && > select && select->cond && > (select->cond->used_tables() & table->map) && > !table->file->pushed_cond) > diff --git a/sql/sql_class.h b/sql/sql_class.h > index f96110e..c42a87a 100644 > --- a/sql/sql_class.h > +++ b/sql/sql_class.h > @@ -597,7 +597,6 @@ typedef struct system_variables > my_bool tx_read_only; > my_bool low_priority_updates; > my_bool query_cache_wlock_invalidate; > - my_bool engine_condition_pushdown; > my_bool keep_files_on_create; > > my_bool old_mode; > diff --git a/sql/sql_priv.h b/sql/sql_priv.h > index 5dc1918..9578c2f 100644 > --- a/sql/sql_priv.h > +++ b/sql/sql_priv.h > @@ -199,7 +199,7 @@ template <class T> bool valid_buffer_range(T jump, > #define OPTIMIZER_SWITCH_INDEX_MERGE_SORT_UNION (1ULL << 2) > #define OPTIMIZER_SWITCH_INDEX_MERGE_INTERSECT (1ULL << 3) > #define OPTIMIZER_SWITCH_INDEX_MERGE_SORT_INTERSECT (1ULL << 4) > -#define OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN (1ULL << 5) > +#define deprecated_ENGINE_CONDITION_PUSHDOWN (1ULL << 5) > #define OPTIMIZER_SWITCH_INDEX_COND_PUSHDOWN (1ULL << 6) > #define OPTIMIZER_SWITCH_DERIVED_MERGE (1ULL << 7) > #define OPTIMIZER_SWITCH_DERIVED_WITH_KEYS (1ULL << 8) > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index 05c88a5..f061b51 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -9561,10 +9561,8 @@ make_join_select(JOIN *join,SQL_SELECT *select,COND *cond) > if (tab->table) > { > tab->table->file->pushed_cond= NULL; > - if (((thd->variables.optimizer_switch & > - OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN) || > - (tab->table->file->ha_table_flags() & > - HA_MUST_USE_TABLE_CONDITION_PUSHDOWN)) && > + if ((tab->table->file->ha_table_flags() & > + HA_MUST_USE_TABLE_CONDITION_PUSHDOWN) && It might be that it is better to use 'new' name for the flag in the lines we have changed. > !first_inner_tab) > { > COND *push_cond= > @@ -23598,10 +23596,8 @@ void JOIN_TAB::save_explain_data(Explain_table_access *eta, table_map prefix_tab > { > const COND *pushed_cond= tab->table->file->pushed_cond; > > - if (((thd->variables.optimizer_switch & > - OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN) || > - (tab->table->file->ha_table_flags() & > - HA_MUST_USE_TABLE_CONDITION_PUSHDOWN)) && > + if ((tab->table->file->ha_table_flags() & > + HA_MUST_USE_TABLE_CONDITION_PUSHDOWN) && The same. > pushed_cond) > { > eta->push_extra(ET_USING_WHERE_WITH_PUSHED_CONDITION); > diff --git a/sql/sys_vars.cc b/sql/sys_vars.cc > index 7f7c725..db764d7 100644 > --- a/sql/sys_vars.cc > +++ b/sql/sys_vars.cc > @@ -2181,13 +2181,15 @@ export const char *optimizer_switch_names[]= > "exists_to_in", > "default", NullS > }; > -/** propagates changes to @@engine_condition_pushdown */ > static bool fix_optimizer_switch(sys_var *self, THD *thd, > enum_var_type type) > { > SV *sv= (type == OPT_GLOBAL) ? &global_system_variables : &thd->variables; > - sv->engine_condition_pushdown= > - MY_TEST(sv->optimizer_switch & OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN); > + if (sv->optimizer_switch & deprecated_ENGINE_CONDITION_PUSHDOWN) > + push_warning_printf(current_thd, Sql_condition::WARN_LEVEL_WARN, > + ER_WARN_DEPRECATED_SYNTAX_NO_REPLACEMENT, > + ER(ER_WARN_DEPRECATED_SYNTAX_NO_REPLACEMENT), > + "engine_condition_pushdown=on"); > return false; > } > static Sys_var_flagset Sys_optimizer_switch( > diff --git a/storage/spider/mysql-test/spider/bg/r/spider_fixes.result b/storage/spider/mysql-test/spider/bg/r/spider_fixes.result > index 6db61fa..3033586 100644 > [skip]

2 1

Re: [Maria-developers] Learning about MariaDB, Getting Started
by Kavneet Kaur 26 Aug '14

26 Aug '14

Hi, James, I am aiming to become am interested in data security and reducing query time by increasing efficiency and computational speed. Coding and optimizing codes are also cup of my tea. How can I contribute this in MariaDB as it's the only open source database software that attracted me. Regards,Kavneet Kaur Date: Fri, 22 Aug 2014 04:44:45 -0700 From: actionmessage(a)yahoo.com Subject: Re: [Maria-developers] Learning about MariaDB, Getting Started To: kavneetk(a)outlook.com; maria-developers(a)lists.launchpad.net; ashish1294(a)gmail.com Hi Kavneet and Ashish. Depends on what you're interested in working on, but here's some suggestions.Direct link to currently open worklogs:https://mariadb.atlassian.net/browse/MDEV-6629?filter=-4Three easy MDEVs and some other areas to work on:- MDEV-6584 - not using tcp_fastopen- MDEV-6555 - sql_log_bin should be SESSION_ONLY- MDEV-5696 - Graceful MariaDB shutdown at Windows restart- test and see if fixes are needed for SSL, Galera, Cassandra, GTID, IPv6 or partition features. "Howto Add a New Command to the MySQL Server" Tutorial http://www.jebriggs.com/blog/2013/05/howto-add-a-new-command-to-the-mysql-s…, James. From: Kavneet Kaur <kavneetk(a)outlook.com> To: "maria-developers(a)lists.launchpad.net" <maria-developers(a)lists.launchpad.net> Sent: Thursday, August 21, 2014 11:49 PM Subject: [Maria-developers] Learning about MariaDB Hi! I am highly interested in subject of database. I am also interested in becoming a research developer. Worked on MS SQL. Want to try MariaDB. May I get the guidance about this product as i'm totally new to it. Is there any specific way to begin? _______________________________________________Mailing list: https://launchpad.net/~maria-developersPost to : maria-developers(a)lists.launchpad.netUnsubscribe : https://launchpad.net/~maria-developersMore help : https://help.launchpad.net/ListHelp

3 6

[Maria-developers] Please review: 10% performance improvement for INSERT INTO t1 (blob) VALUES ('long value')
by Alexander Barkov 26 Aug '14

26 Aug '14

Hi, I think I found and fixed a bottleneck in how string literals are processed. The attached patch improves INSERT performance for strings, and especially for long blob and text values. Insertion time for 10MByte blob 1024 times (10GByte of data total) reduced on my laptop from 300 seconds to 274 seconds, which gives near 10% pure time improvement. What the patch changes: Before the patch : 1. The function get_text() is sql_lex.cc allocated an unescaped copy of every string literal (unescaping backslashes and double quotes, if any). Strangely, copying happened even if there were no really any escapes in the string. 2, The syntax parser in sql_yacc.yy created Item_string using the new unescaped buffer. Furthermore, in case of a multi-byte connection character set, the constructor for Item_string made another loop on the unescaped buffer, to calculate length in characters, which is needed to set max_length properly. After the patch: Item is now created using directly the SQL fragment, without making a copy. Even for escaped values!!! Length in characters is also calculated during the very first pass in get_text(), without any additional loops in the Item constructors. Unescaping is now done in the very end, when the value is actually needed: - Either in Field::store(). This happens when the string value is used in: INSERT INTO t1 VALUES('string'); Unescaping is done directly to the Field buffer, without any intermediary temporary storage. - Or in val_str(), if the string value is used elsewhere (in SELECT list, functions, operators, etc). The unescaped value is cached, to make sure that unescaping is done only one time, even if the value is needed for multiple rows, like in: SELECT * FROM t1 WHERE a='string with backslash or quote escapes'; The patch also fixes a few bugs related to string conversion/copying (some of them were reported on JIRA earlier, and some of them I found while working on the patch). Please also find "a.c" attached. I used it to test the improvement. I used a RelWithDebInfo compiled binary for tests. Note, "perf" and "valgrind --tool=cachegrind" also demonstrate a good improvement (reduced CPU and memory usage). I can send their statistics in a separate letter, if anyone is interested. Please review the patch. Thanks.

1 0

[Maria-developers] List affecting CVEs at mariadb.com
by Otto Kekäläinen 26 Aug '14

26 Aug '14

Hello Daniel (and others), The usual changelogs[1] and relese notes[2] don't seem to contain CVE identifiers, or even a separate section about fixed security issues For the downstream security teams if would be reassuring if the CVE information would be easily available. For example if the security teams follow the CVE news and they for example know or suspect that CVE-2014-4260 affects MariaDB, it would be nice to see if it is already fixed or what version it was fixed in, so downstream security teams can organize and prioritize their patching and release work. Do you have any suggestion how to address this? Should we maybe have a separate wiki page, e.g. https://mariadb.com/kb/en/mariadb/cve/ that would have a table of CVEs and MariaDB 5.5/10.0/Galera versions where they are fixed? Or should just each release notes include a subsection "Security" with these details? Something else? Of course we need to consider timing issues, e.g. a security issue fixed in MariaDB might get publicity and a CVE only later when Oracle releases it, and in those cases old release notes need to be upgraded to include the CVE identifiers. [1] https://mariadb.com/kb/en/mariadb-10013-changelog/ [2] https://mariadb.com/kb/en/mariadb-10013-release-notes/ (To be exact, googling for 'mariadb cve' does give one hit at mariadb.com in the 5.3.12 release notes)

3 3

Re: [Maria-developers] [Commits] Rev 4271: MDEV-6601 Assertion `!thd->in_active_multi_stmt_transa ction() || thd->in_multi_stmt_transaction_mode()' failed on executing a stored procedure with commit in lp:~maria-captains/maria/5.5
by Alexey Botchkov 25 Aug '14

25 Aug '14

Hi, Sergei. I'd add SERVER_STATUS_LAST_ROW_SENT to the status_backup_mask. It's not necessary as I don't see it affecting anything, but just to make a sign that we don't want this status to go higher out of the call. Best regards. HF 25.08.2014 10:49, Sergei Golubchik wrote: > At lp:~maria-captains/maria/5.5 > > ------------------------------------------------------------ > revno: 4271 > revision-id: sergii(a)pisem.net-20140825064904-h8isjhv963kbxlzy > parent: knielsen(a)knielsen-hq.org-20140813134639-wk760plnzg5wu4x8 > fixes bug: https://mariadb.atlassian.net/browse/MDEV-6601 > committer: Sergei Golubchik <sergii(a)pisem.net> > branch nick: 5.5 > timestamp: Mon 2014-08-25 08:49:04 +0200 > message: > MDEV-6601 Assertion `!thd->in_active_multi_stmt_transa ction() || thd->in_multi_stmt_transaction_mode()' failed on executing a stored procedure with commit > > Don't restore the whole of thd->server_status after a routine invocation, > only restore SERVER_STATUS_CURSOR_EXISTS, as --ps --embedded needs. > In particular, don't restore SERVER_STATUS_IN_TRANS. > === modified file 'mysql-test/r/sp-bugs.result' > --- a/mysql-test/r/sp-bugs.result 2014-01-24 12:50:18 +0000 > +++ b/mysql-test/r/sp-bugs.result 2014-08-25 06:49:04 +0000 > @@ -268,3 +268,9 @@ END $$ > CALL test_5531(1); > DROP PROCEDURE test_5531; > DROP TABLE t1; > +create procedure sp() begin > +commit; > +end| > +start transaction; > +call sp(); > +drop procedure sp; > > === modified file 'mysql-test/t/sp-bugs.test' > --- a/mysql-test/t/sp-bugs.test 2014-01-24 12:50:18 +0000 > +++ b/mysql-test/t/sp-bugs.test 2014-08-25 06:49:04 +0000 > @@ -285,3 +285,16 @@ DELIMITER ;$$ > CALL test_5531(1); > DROP PROCEDURE test_5531; > DROP TABLE t1; > + > +# > +# MDEV-6601 Assertion `!thd->in_active_multi_stmt_transa ction() || thd->in_multi_stmt_transaction_mode()' failed on executing a stored procedure with commit > +# > +delimiter |; > +create procedure sp() begin > + commit; > +end| > +delimiter ;| > +start transaction; > +call sp(); > +drop procedure sp; > + > > === modified file 'sql/sp_head.cc' > --- a/sql/sp_head.cc 2014-04-15 13:17:47 +0000 > +++ b/sql/sp_head.cc 2014-08-25 06:49:04 +0000 > @@ -1224,6 +1224,7 @@ sp_head::execute(THD *thd, bool merge_da > Item_change_list old_change_list; > String old_packet; > uint old_server_status; > + const uint status_backup_mask= SERVER_STATUS_CURSOR_EXISTS; > Reprepare_observer *save_reprepare_observer= thd->m_reprepare_observer; > Object_creation_ctx *saved_creation_ctx; > Warning_info *saved_warning_info; > @@ -1358,7 +1359,7 @@ sp_head::execute(THD *thd, bool merge_da > It is probably safe to use same thd->convert_buff everywhere. > */ > old_packet.swap(thd->packet); > - old_server_status= thd->server_status; > + old_server_status= thd->server_status & status_backup_mask; > > /* > Switch to per-instruction arena here. We can do it since we cleanup > @@ -1488,7 +1489,7 @@ sp_head::execute(THD *thd, bool merge_da > thd->spcont->pop_all_cursors(); // To avoid memory leaks after an error > > /* Restore all saved */ > - thd->server_status= old_server_status; > + thd->server_status= (thd->server_status & ~status_backup_mask) | old_server_status; > old_packet.swap(thd->packet); > DBUG_ASSERT(thd->change_list.is_empty()); > old_change_list.move_elements_to(&thd->change_list); > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] Getting Started
by Ashish Kedia 23 Aug '14

23 Aug '14

Hi all, I am 3rd Year undergraduate student majoring in Information Technology from India. I am new to MariaDB however I wish to contribute. I am well acquainted with C++ and other tools required. Can someone help me find small tasks/bugs/projects to get started with ? Regards, Ashish Kedia, 3rd Year Undergraduate Student, Dept. of Information Technology, National Institute of Technology Karnataka, Surathkal

3 2

[Maria-developers] Closing bugs as Incomplete/Cannot reproduce requires FixVersion?
by Sergey Petrunia 23 Aug '14

23 Aug '14

Hi Serg, (writing to you because I think it was you who set Jira's rules for Fix Version field). I'm trying to close MDEV-6032 as "Incomplete" ("Cannot reproduce" would also work). This resolution is meant to say that the issue is not really resolved. And Jira's "Close Issue" dialog gives me this error: Field Fix Version/s with actual value does not match regular expression \[([0-9]+\.[0-9]+\.[0-9]+.*|N/A)\] I think it doesn't make sense. I even think that "Fix Version" must be empty for bugs that are closed with resolution other than "Fixed" or "Duplicate". Do you agree with this reasoning? If yes, is it possible to fix Jira to follow this logic? BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

2 1

[Maria-developers] Learning about MariaDB
by Kavneet Kaur 22 Aug '14

22 Aug '14

Hi! I am highly interested in subject of database. I am also interested in becoming a research developer. Worked on MS SQL. Want to try MariaDB. May I get the guidance about this product as i'm totally new to it. Is there any specific way to begin?

1 0

[Maria-developers] [GSoC] Pencils down
by sriram patil 21 Aug '14

21 Aug '14

Hi All, Following is the link to my pencils down blog post. You can go through it even if you have not followed my blog posts before. http://goo.gl/DqBgDe Thanks a lot for all your feedback and suggestions. Sriram.

3 3

[Maria-developers] [GSOC2014] UseServerPrepStmts - Connector/J
by Puneet Dewan 20 Aug '14

20 Aug '14

Hi Everyone, I have pushed the code and it can be found on the following link: https://code.launchpad.net/~team-java-connector/mariadb-java-client/mariadb… Thanks for the support. :) Please review and give suggestions. Regards Puneet.

1 0

Re: [Maria-developers] [GSoC] Optimize mysql-test-runs - Results of new strategy
by Pablo Estrada 19 Aug '14

19 Aug '14

Hello Elena and all, First, addressing the previous email: Looking at the dump, I see it can also happen that the dump contains > several records for a pair platform/bbnum. I am not sure why it happens, I > think it shouldn't, might be a bug in buildbot and/or configuration, or > environmental problems. Anyway, due to the way we store output files, they > can well override each other in this case, thus for several platform/bbnum > record you will have only one file. I suppose that's what was hard to > resolve, sorry about that. > No worries ; ). There are several cases where platform and build number are the same. The system just names the files as follows: <platform>_<build_id>-log-test_1-stdio <platform>_<build_id>-log-test_2-stdio ..... <platform>_<build_id>-log-test_5-stdio These files seem to correspond temporarily with the test runs (*test_1-stdio) belongs to the first test_run of the same plt-bnum, and so on. Unfortunately, there are some cases where there are more test_runs on the dump than files available, and this means that it's impossible to be sure which file belongs to which test_run exactly. > You should consider skipped tests, at least for now. Your logic that they > are skipped because they can't be run is generally correct; unfortunately, > MTR first produces the *full* list of tests to run, and determines whether > a test can be run or not on a later stage, when it starts running the > tests. Your tool will receive the initial test list, and I'm not sure it's > realistic to re-write MTR so that it takes into account limitations that > cause skipping tests before creating the list. > I see. Okay then, duly noted. Possibly it's better to skip a test run altogether if there is no input > list for it; it would be definitely the best if there were 5K (or whatever > slice you are currently using) of continuous test runs with input lists; if > it so happens that there are lists for some branches but not others, you > can skip the branch entirely. > This doesn't seem like a good option. Recall drops seriously, and the test_runs that have a corresponding file don't seem to have a special pattern, and tend to have long spaces between them, so the information becomes irrelevant, and seemingly, not useful. > The core module should take as parameters > - list of tests to choose from, > - size of the running set (%), > - branch/platform (if we use them in the end), > and produce a new list of tests of the size of the running set. > > The wrapper module should > - read the list of tests from the outside world (for now, from a file), > - receive branch/platform as command-line options, > - have the running set size set as an easily changeable constant or as a > configuration parameter, > > and return the list of tests -- lets say for now, in the form of <test > suite>.<test name>, blank-separated, e.g. > main.select innondb.create-index ... > >> >> I am almost done 'translating' the code into a solution that divides it in 'core' and 'wrapper'. There are a few bugs that I still haven't figured out, but I believe I can iron those out pretty soon. I will also incorporate the percentage rather than fixed running_set. Now, regarding the state of the project (and the recall numbers that I am able to achieve so far), here are some observations: - Unfortunately, I am running out of ideas to try to improve recall. I tried tuning some parameters, giving more weight to ones or others, etc. I still wasn't able to push recall beyond ~87% on the strategy that uses file correlations. For what I've seen, some failures are just extremely hard to predict. - The strategy that uses only a weighted average of the failure frequency achieves a higher recall, but for a shorter time. The recall decays quickly afterwards. I may try to add some file-correlations to this strategy, to see if the recall can be sustained for a longer term. - There is one problem that I see regarding the data and the potential real-world implementation of the program: By verifying the recall with the historical data, we run the possibility of 'expecting' overfitting... so the results regarding the errors found when comparing to the historical data, and the results that could have been obtained by a real-world implementation are potentially different. A possible way to address that issue would require modifying the buildbot to gather more data over a longer term. So having said that, I am looking for some advice in the following regards: - I will try to take a step back from the new strategy, and see how I can adapt the original strategy to prevent the recall function from declining so sharply with time. - I will also spend some time keeping a codebase that adjusts better to the model that we need for the implementation. I will upload code soon. All suggestions are welcome. - Nonetheless, I feel that more data would allow to improve the algorithm greatly. Is it possible to prepare logging into the buildbot that would allow for more precise data collection? A slower, more iterative process, working closer with the buildbot and doing more detailed data collection might deliver better results. (I understand that this would probably influence the time-scope of the project) Let me know what you think about my suggestions. Regards Pablo

2 22

[Maria-developers] [GSoC] Week Twelve!
by sriram patil 14 Aug '14

14 Aug '14

Hi All, Following is the link to my twelfth weeks blog post. http://goo.gl/Zkr86e Suggestions and/or reviews are welcome! Thanks, Sriram

3 2

[Maria-developers] Fwd: I have a replication patch, please accept it.
by Colin Charles 12 Aug '14

12 Aug '14

Xu Liang, the best place to post it is maria-developers I've done that via a FWD -- Kristian, what do you think? Begin forwarded message: > From: "Xu,Liang(OP)" <xuliang02(a)baidu.com> > Subject: I have a replication patch,please accept it. > Date: 6 August 2014 15:31:27 GMT+8 > To: "kb-admins(a)askmonty.org" <kb-admins(a)askmonty.org> > > http://bugs.mysql.com/bug.php?id=73479 > > > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > Baidu DBA 徐良 > 分机：86912826 > 手机：18689220441 > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Colin Charles, Chief Evangelist, SkySQL - The MariaDB Company blog: http://bytebot.net/blog/| t: +6-012-204-3201 | Skype: colincharles

1 0

[Maria-developers] MariaDB 10.0.13 Release Prep
by Daniel Bartholomew 11 Aug '14

11 Aug '14

I've begun the prep work for the MariaDB 10.0.13 release. Draft changelog and release notes are here: - https://mariadb.com/kb/en/mariadb-10013-release-notes/ - https://mariadb.com/kb/en/mariadb-10013-changelog/ A big thanks to those that have already contributed content! As always, please let me know ASAP if anything else needs to be added/changed/etc..., or jump in and do it yourself. Thanks! -- Daniel Bartholomew, MariaDB Release Manager MariaDB | http://mariadb.com

1 0

[Maria-developers] innodb-buffer-pool-instances confusion
by Daniel Bartholomew 07 Aug '14

07 Aug '14

I was chatting with some folks on IRC and someone pointed this out to me. On the "Upgrading From MariaDB 5.5 to MariaDB 10.0" page, it says that the new default value for innodb-buffer-pool-instances is '0': https://mariadb.com/kb/en/upgrading-from-mariadb-55-to-mariadb-100/ And when I run the following, it says the default is indeed '0': mysqld --no-defaults --verbose --help | grep innodb-buffer-pool-instances ... innodb-buffer-pool-instances 0 But when I'm in the client, I get the following: MariaDB [(none)]> select @@innodb_buffer_pool_instances; +--------------------------------+ | @@innodb_buffer_pool_instances | +--------------------------------+ | 8 | +--------------------------------+ So when innodb-buffer-pool-instances is set to '0', does that mean the actual value is dynamic or calculated in some way based on my other settings? Just trying to understand what's going on, and once I do, use it to update the XtraDB/InnoDB buffer-pool page (which doesn't appear to have been updated for 10.0). https://mariadb.com/kb/en/xtradbinnodb-buffer-pool/ Thanks! -- Daniel Bartholomew, MariaDB Release Manager MariaDB | http://mariadb.com

4 3

[Maria-developers] CONNECT system variables
by Ian Gilfillan 07 Aug '14

07 Aug '14

Hi I am trying to document the CONNECT system variables at https://mariadb.com/kb/en/mariadb/documentation/storage-engines/connect/con… but I can't seem to find much documentation in the code. I managed to put something together from Olivier's notes in JIRA for some of them, but can anyone help fill in the gaps and send me details on the other variables? thanks, ian

1 0

[Maria-developers] a question about group commit
by nanyi607rao 07 Aug '14

07 Aug '14

Hi, as we know there are 3 steps in XA transaction committing 1, prepare step 2, write binary log 3, commit step in engines all these steps need a fsync(). Group commit strategy can make a group of transactions durable with one fsync() at step 2 and step 3, which can lead to dramatic performance enchance. But in step 1, each transaction still do its own fsync(). so why not make several transactions durable whith one fsync() in prepare step just like step 2 and 3, which I think can improve performanc further more ? Thanks 2014-08-04 nanyi607rao

2 3

[Maria-developers] [GSoC] Week Eleven!
by sriram patil 05 Aug '14

05 Aug '14

Hi All, Following is the link to the blog post for my eleventh week into GSoC http://goo.gl/TN3U0K Suggestions and/or reviews are welcome. Thanks, Sriram

3 2

[Maria-developers] MariaDB 5.5.39 Release Prep
by Daniel Bartholomew 05 Aug '14

05 Aug '14

All, I've begun prep work on the MariaDB 5.5.39 release. Draft release notes and changelog are here: - https://mariadb.com/kb/en/mariadb-5539-changelog/ - https://mariadb.com/kb/en/mariadb-5539-release-notes/ Please add any necessary or notable items to the release notes ASAP. Current plan is to activate/announce the release on 5 Aug. Thanks! -- Daniel Bartholomew, MariaDB Release Manager MariaDB | http://mariadb.com

1 0

[Maria-developers] Connector Coding till 5th Aug
by Puneet Dewan 05 Aug '14

05 Aug '14

Hi Everyone, Last week, 1.I was working on the Binary Protcol Resultset which is still going on. 2.Added code for Batch Statements in MySQlServerSidePreparedStatement. 3.Added utility methods in ParameterWriter.java for sending different parameters in Binary Protocol format. 4.Code review with Massimo This week, 1.Complete the Binary protocol resultset. 2.Test cases for batch statements and allowmultiqueries. 3.Code review with Massimo. Regards Puneet.

1 0

[Maria-developers] [GSoC] Introduction Mail
by Anshu Avinash 04 Aug '14

04 Aug '14

Hi all, I have been accepted for the project "Self-Tuning Optimizer". I will be doing this project under the mentorship of Sergei Golubchik. There is already a task created which will track the progress for this project: https://mariadb.atlassian.net/browse/MDEV-350 . I'll start with getting familiar with the code base, reading some literature on cost estimation in optimizer and interacting with community. About me: I am currently a 4th year student pursuing B.Tech - M.Tech dual degree in Computer Science and Engineering from Indian Institute of Technology (IIT) – Kanpur, India. This would be my first GSoC project. I had done an internship at Facebook last year. I'll also be maintaining a blog at http://igniting.in . I'm really excited to be a part of mariadb and I hope to make a significant contribution to it. Regards Anshu Avinash

5 48

[Maria-developers] Review request MDEV-6274 Collation usage statistics (for feedback plugin)
by Alexander Barkov 30 Jul '14

30 Jul '14

Hello Sergei, Please review a patch for MDEV-6274. Thanks.

2 3

Re: [Maria-developers] [Commits] Rev 4227: MDEV-6441: memory leak in file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6441/
by Igor Babaev 29 Jul '14

29 Jul '14

Sanja, Correct the comment for the commit and push. The only question I have concerns another comment: +# MDEV-5515: sub-bug nets of 3rd execution crash ^ Is this an intended word? Regards, Igor. On 07/28/2014 05:36 AM, sanja(a)askmonty.org wrote: > At file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6441/ > > ------------------------------------------------------------ > revno: 4227 > revision-id: sanja(a)askmonty.org-20140728123541-3dh3f20p41iyrefm > parent: sanja(a)askmonty.org-20140715093734-42nrug10i0xw8hvb > committer: sanja(a)askmonty.org > branch nick: work-maria-5.5-MDEV-6441 > timestamp: Mon 2014-07-28 15:35:41 +0300 > message: > MDEV-6441: memory leak > > Whole mysql_derived_prepare() was executed on statement memory. Now it is executed on runtime meory and all bugs induced bu this fixed. mysql_derived_prepare() was executed on the statement memory. Now it is executed on the runtime memory. All bugs induced by this were fixed. > > > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] Performance: question about strings created in sql_lex.cc
by Alexander Barkov 29 Jul '14

29 Jul '14

Hello, When parsing an SQL query, the function get_text() in sql_lex.cc always makes a copy of the string, allocating new memory using lip->m_thd->alloc() and copying the original string from the SQL query fragment to this new memory. Note, this happens even if the string does not have any escaped characters! I'm seeing some performance bottleneck here. It would be nice to avoid copying. Why could not we point a non-escaped string to the original SQL query fragment where the string is actually written? Are there any situations when the original SQL query is not accessible any more, but Item_string created at parse time is still in use? Are there any other things that are created in sql_yacc.yy using the TEXT_STRING token, which could exist longer than the original SQL query? Thanks.

1 0

[Maria-developers] merge 10.0-connect -> 10.0
by Alexander Barkov 29 Jul '14

29 Jul '14

Hello Olivier, I tried to merge 10.0-connect into the main 10.0, but two mtr tests failed in the merged tree. Note, they don't fail in 10.0-connect, so I guess something have changed in the main tree which makes your latest changes in 10.0-connect incompatible. Please find the log attached. I have no idea how to fix the failures. Can I push the merge into 10.0-connect, so you can fix the problems? Also, I noticed that you changed the sources outside of the "/storage/connect" directory: > --- sql/sql_partition.cc 2014-03-26 21:25:38 +0000 > +++ sql/sql_partition.cc 2014-07-29 04:44:43 +0000 > @@ -3195,13 +3195,26 @@ uint32 get_partition_id_cols_list_for_en > (list_index - 1)*num_columns, > nparts, left_endpoint, > include_endpoint))); > - > +#if 0 > if (!left_endpoint) > { > /* Set the end after this list tuple if not already after the last. */ > if (list_index < part_info->num_parts) > list_index++; > } > +#else > + if (!left_endpoint && list_index < part_info->num_list_values) > + { > + /* > + Set the end after this list tuple if it is not already after the last > + and it matches. > + */ > + int cmp = cmp_rec_and_tuple_prune(list_col_array + list_index*num_columns, > + nparts, left_endpoint, include_endpoint); > + if (cmp >= 0) > + list_index++; > + } > +#endif > > DBUG_RETURN(list_index); > } Can you please comment why do you need this change? Did you discuss this change with anyone? Thanks.

1 0

[Maria-developers] Connector Coding till 28th July
by Puneet Dewan 28 Jul '14

28 Jul '14

Hi Everyone, Last week, 1.Migrated the code from MySQLPreparedStatement to MySQLServerSidePreparedStatement. 2.Removed duplicate code in classes like IntParameter, DoubleParameter, LongParameter and used methods of class ParameterWriter(provides utility methods) in the above classes. 3.Did the analysis of TextProtocol Resultset has been implemented in connector code and find a way on how to write for Binary Protocol Resultset. 4.Fixed the new_params_bound_flag issue. 5.Code review with Massimo. This week, 1.Code review with Massimo 2.Complete the coding for Binary Protocol Resultset. 3.Write code for allowMultiQueries, Batchstatements. Regards Puneet.

1 0

[Maria-developers] [GSoC] Week Nine and Ten!
by sriram patil 28 Jul '14

28 Jul '14

Hi All, Following is the blog link about my ninth and tenth week into GSoC. http://goo.gl/N5Le32 Suggestions and/or reviews are welcome! Thanks, Sriram

1 0

[Maria-developers] MDEV-4511 Assertion `scale <= precision' fails on GROUP BY TIMEDIFF with incorrect types
by Alexander Barkov 25 Jul '14

25 Jul '14

Hello Sergei, Please review a new version of the patch for MDEV-4511, it also fixes more crashing problems found, as well as non-crashing problems reported in MDEV-6302 I removed find_num_type() and fix_num_length_and_dec() which were hard to track, and for some Items these methods were erroneously called twice. fix_num_length_and_dec() now only exists as a non-virtual method in Item_sum and Item_udf_func. The patch is for 5.3, but it is still valid for 10.0, as we still have temporal functions with NOT_FIXED_DEC in 10.0. Thanks.

2 3

[Maria-developers] Any plans about starting to use /var/lib/mariadb for data instead of /var/lib/mysql?
by Otto Kekäläinen 24 Jul '14

24 Jul '14

Hello! Does the MariaDB project have any plans about switching the data dir to /var/lib/mariadb in some release? In Debian there has been discussion about separating the binary-incompatible MySQL forks into separate /var/lib/<fork> to ease maintenance and secure data so that purging some packages will not accidentally delete data that another package uses. The thread is named "Discussion on handling 5.6 related matters" and available at http://lists.alioth.debian.org/pipermail/pkg-mysql-maint/2014-July/thread.h… I'd appreciate any guidelines here on what the upstream plans are. The Debian MySQL/MariaDB packaging team willl have an IRC meeting tomorrow at 16:00 UTC and I need to express my opinion on the mattes and what kind of changes I'm willing to do to the MariaDB package. Obviously I'd like to follow upstream as much as possible to keep the delta between Debian and upstream small and thus the maintenance burden reasonable. -- Check out our blog at http://seravo.fi/blog and follow @ottokekalainen

3 2

[Maria-developers] please review mdev-5745 analyze MySQL fix for bug#12368495
by Alexander Barkov 23 Jul '14

23 Jul '14

Hi Serg, please review a patch for mdev-5745. Thanks.

1 0

Re: [Maria-developers] [Commits] 00ed367: MDEV-4262 - P_S discovery
by Sergei Golubchik 22 Jul '14

22 Jul '14

Hi, Sergey! On Jul 07, svoj(a)mariadb.org wrote: > revision-id: 00ed36700addfbce91940223cce4ccd6ee5823a1 > parent(s): f37f19a6c790100cc2ae5f483ba4db34a517f0ae > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2014-07-07 14:32:13 +0400 > message: > > MDEV-4262 - P_S discovery > > Discover P_S tables automatically. > > Most of this patch is code clean-up: > - removed tests and code responsible for P_S tables correctness verification > - always return error from ha_perfschema::create() > - install/upgrade scripts won't create P_S tables anymore Great work, thanks! The patch is pretty much "ok to push", but, first I want to understand why you deleted table_file_summary.cc file. See other minor comments below. > diff --git a/mysql-test/suite/perfschema/include/upgrade_check.inc b/mysql-test/suite/perfschema/include/upgrade_check.inc > index 52d4cfd..1532de8 100644 > --- a/mysql-test/suite/perfschema/include/upgrade_check.inc > +++ b/mysql-test/suite/perfschema/include/upgrade_check.inc > @@ -3,7 +3,6 @@ > # > > --source include/count_sessions.inc > ---error 1 > --exec $MYSQL_UPGRADE --skip-verbose --force > $out_file 2> $err_file > --source include/wait_until_count_sessions.inc The rest of this upgrade_check.inc file does (according to the comment) "Verify that mysql_upgrade complained about the performance_schema" perhaps you should change the comment to say "does not complain" ? > > diff --git a/scripts/mysql_performance_tables.sql b/scripts/mysql_performance_tables.sql > index d495578..3061603 100644 > --- a/scripts/mysql_performance_tables.sql > +++ b/scripts/mysql_performance_tables.sql > @@ -53,1427 +53,3 @@ SET @str = IF(@broken_pfs = 0, @cmd, 'SET @dummy = 0'); > PREPARE stmt FROM @str; > EXECUTE stmt; > DROP PREPARE stmt; Why did you keep the above? I'd think that a simple DROP DATABASE IF EXISTS performance_schema should be enough to replace the whole file. And it could be moved back to mysql_system_tables.sql, so this file could be completely removed. > - > -# > -# From this point, only create the performance schema tables > -# if the server is built with performance schema > -# > - > -set @have_pfs= (select count(engine) from information_schema.engines where engine='PERFORMANCE_SCHEMA' and support != 'NO'); > - > -# > -# TABLE COND_INSTANCES > -# ... > diff --git a/storage/perfschema/pfs_engine_table.cc b/storage/perfschema/pfs_engine_table.cc > index 958a2bd..6f0a363 100644 > --- a/storage/perfschema/pfs_engine_table.cc > +++ b/storage/perfschema/pfs_engine_table.cc > @@ -1440,5 +1319,17 @@ end: > DBUG_RETURN(false); > } > > +int pfs_discover_table_names(handlerton *hton, LEX_STRING *db, > + MY_DIR *dir, hm, no "unused argument" compiler warnings? okay... > + handlerton::discovered_list *result) > +{ > + if (compare_table_names(db->str, PERFORMANCE_SCHEMA_str.str)) > + return 0; > + for (size_t i= 0; i < array_elements(all_shares) - 1; i++) > + result->add_table(all_shares[i]->m_name.str, > + all_shares[i]->m_name.length); > + return 0; > +} > + > /** @} */ > > diff --git a/storage/perfschema/table_file_summary.cc b/storage/perfschema/table_file_summary.cc > deleted file mode 100644 > index 104fa0f..0000000 > --- a/storage/perfschema/table_file_summary.cc > +++ /dev/null > @@ -1,372 +0,0 @@ Why did you remove this file? > -/* Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. > - > - This program is free software; you can redistribute it and/or modify > - it under the terms of the GNU General Public License as published by > - the Free Software Foundation; version 2 of the License. Regards, Sergei

2 4

Re: [Maria-developers] MDEV-4904 - Add query cache hit information to Percona Response time distribution plugin (MDEV-4568)
by Sergei Golubchik 22 Jul '14

22 Jul '14

Hi, svoj! On Jun 26, svoj(a)mariadb.org wrote: > revision-id: f37f19a6c790100cc2ae5f483ba4db34a517f0ae > parent(s): 4a3f0684195703996b23c51af79cf946b85b8053 > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2014-06-26 16:37:46 +0400 > message: > > MDEV-4904 - Add query cache hit information to Percona Response time distribution plugin (MDEV-4568) > > Added two new fields to INFORMATION_SCHEMA.QUERY_RESPONSE_TIME: > QC_HITS_COUNT - query cache hits count > QC_HITS_TOTAL - cumulative response time for query cache hits. This is somewhat questionable feature. Queries served from qc don't take much time, which means that @@query_response_time_range_base should be different for them. Otherwise all qc hits will be in the first few slots. But a different base means a different table (QUERY_CACHE_RESPONSE_TIME ?) and a different set of configuration variables. On the other hand, this current implementation is much simpler. So I think we can accept this patch. But I don't know how useful it'll be. Alternatively, you can do a separate I_S table for QC. As you like. > diff --git a/plugin/query_response_time/mysql-test/query_response_time/query_response_time-stored.result b/plugin/query_response_time/mysql-test/query_response_time/query_response_time-stored.result > index bec7007..e487fc5 100644 > --- a/plugin/query_response_time/mysql-test/query_response_time/query_response_time-stored.result > +++ b/plugin/query_response_time/mysql-test/query_response_time/query_response_time-stored.result > @@ -40,51 +40,51 @@ SHOW GLOBAL VARIABLES where Variable_name like 'QUERY_RESPONSE_TIME_RANGE_BASE'; > Variable_name Value > query_response_time_range_base 2 > SELECT * FROM INFORMATION_SCHEMA.QUERY_RESPONSE_TIME; I'd rather replaced * with TIME,COUNT,TOTAL - less changes and more future proof. > -TIME COUNT TOTAL > - 0.000001 45 0.000000 > +TIME COUNT TOTAL QC_HITS_COUNT QC_HITS_TOTAL > + 0.000001 45 0.000000 0 0.000000 ... Regards, Sergei

1 0

[Maria-developers] Why does name resolution stop in crash stack trace?
by Sergey Petrunia 21 Jul '14

21 Jul '14

Hi, Look at today's crash report in maria-discuss@. A question not related to the cause of that crash: why do stack traces that go inside the storage engine cannot be resolved? Is something wrong with our build process? On Fri, Jun 13, 2014 at 08:15:39PM -0700, grantksupport(a)operamail.com wrote: > ==> /var/log/mariadb/mariadb-err.log <== > 140613 20:07:24 [ERROR] mysqld got signal 11 ; ... > Server version: 10.0.11-MariaDB-log > key_buffer_size=268435456 > read_buffer_size=1048576 > max_used_connections=1 > max_threads=66 > thread_count=1 > It is possible that mysqld could use up to > key_buffer_size + (read_buffer_size + > sort_buffer_size)*max_threads = 601410 K bytes of > memory > Hope that's ok; if not, decrease some variables in the > equation. > > Thread pointer: 0x0x7fcf132fa008 > Attempting backtrace. You can use the following > information to find out > where mysqld died. If you see no messages after this, > something went > terribly wrong... > stack_bottom = 0x7fcf4a6e4e78 thread_stack 0x100000 > /usr/local/mariadb/bin/mysqld(my_print_stacktrace+0x2e)[0xacb60e] > /usr/local/mariadb/bin/mysqld(handle_fatal_signal+0x390)[0x6f5a10] > /lib64/libpthread.so.0(+0xf9f0)[0x7fcf4ff2a9f0] > /usr/local/mariadb/bin/mysqld[0x915cba] > /usr/local/mariadb/bin/mysqld[0x909e27] > /usr/local/mariadb/bin/mysqld[0x90a35b] > /usr/local/mariadb/bin/mysqld[0x90c759] > /usr/local/mariadb/bin/mysqld[0x9169d5] > /usr/local/mariadb/bin/mysqld[0x8aaf2b] > /usr/local/mariadb/bin/mysqld[0x8ab3cf] > /usr/local/mariadb/bin/mysqld[0x8ab95a] > /usr/local/mariadb/bin/mysqld[0x895e80] > /usr/local/mariadb/bin/mysqld[0x89643f] > /usr/local/mariadb/bin/mysqld[0x9657da] > /usr/local/mariadb/bin/mysqld[0x965a16] > /usr/local/mariadb/bin/mysqld[0x96a51d] > /usr/local/mariadb/bin/mysqld[0x96ac28] > /usr/local/mariadb/bin/mysqld[0x82de54] > /usr/local/mariadb/bin/mysqld(_ZN7handler9ha_createEPKcP5TABLEP14HA_CREATE_INFO+0x70)[0x6fcbf0] > /usr/local/mariadb/bin/mysqld(_Z15ha_create_tableP3THDPKcS2_S2_P14HA_CREATE_INFOP34st_mysql_const_unsigned_lex_string+0x220)[0x6fd560] > /usr/local/mariadb/bin/mysqld(_Z16rea_create_tableP3THDP34st_mysql_const_unsigned_lex_stringPKcS4_S4_P14HA_CREATE_INFOP7handlerb+0xbb)[0x659d1b] > /usr/local/mariadb/bin/mysqld[0x62e50d] > /usr/local/mariadb/bin/mysqld(_Z26mysql_create_table_no_lockP3THDPKcS2_P14HA_CREATE_INFOP10Alter_infoPbi+0xd3)[0x62ee13] > /usr/local/mariadb/bin/mysqld(_Z18mysql_create_tableP3THDP10TABLE_LISTP14HA_CREATE_INFOP10Alter_info+0xb6)[0x62ef76] > /usr/local/mariadb/bin/mysqld(_Z21mysql_execute_commandP3THD+0x672d)[0x5be04d] > /usr/local/mariadb/bin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x1d7)[0x5bf4f7] > /usr/local/mariadb/bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x15d7)[0x5c1057] > /usr/local/mariadb/bin/mysqld(_Z24do_handle_one_connectionP3THD+0x20c)[0x66f0ec] > /usr/local/mariadb/bin/mysqld(handle_one_connection+0x38)[0x66f138] > /usr/local/mariadb/bin/mysqld[0x9d5184] > /lib64/libpthread.so.0(+0x80db)[0x7fcf4ff230db] > /lib64/libc.so.6(clone+0x6d)[0x7fcf4ebd390d] > > Trying to get some variables. > Some pointers may be invalid and cause the dump to > abort. > Query (0x7fcf1143d020): CREATE TABLE menu_router ( BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

2 1

[Maria-developers] [GSoC] Week Nine!
by sriram patil 21 Jul '14

21 Jul '14

Hi All, There will be a little delay in the ninth week blog post. I need to get the code reviewed first to make the final changes. You can check the commit here <https://github.com/srirampatil/server/commit/3611bfc644714b797acff8b9884e7d…> though. I have also fixed MDEV-6409 <https://mariadb.atlassian.net/browse/MDEV-6409> and it is under review too. Regards, Sriram

1 0

[Maria-developers] Java Connector Coding till 20th July
by Puneet Dewan 20 Jul '14

20 Jul '14

Hi Everyone, Sorry,I did not update in the last week. It is because of my convocation and was meeting my parents after one year in Singapore, so enjoyed a small vacation of 5 days with them. I had code review and discussion with Massimo in that week. But I thought it is better to update next week. Things done: 1.I had initially written code in MySQLPreparedStatement.java but the code was getting ugly so as per recommnedation from Vladislav,I also discussed with my mentor.So we thought of using MySQLServerSidePreparedStatement as this will make code more cleaner. Now prepareStatement method in MySQLConnection will become the factory method to provide either MySQLPreparedStatement or MySQLServerSidePreparedStatement instance. I have migrated my code to MySQLServerSidePreparedStatement. 2.Test case for determining the type of instance, MySQLPreparedStatement or MySQLServerSidePreparedStatement. Next plan: 1.Write executeQuery and executeUpdate methods 2.For executeQuery, need to decide how I can use BinaryProtocolResultset to fetch results from a select query. 3.Write methods for batchstatements and multiqueries. Regards Puneet.

1 0

[Maria-developers] [GSoC] Optimize mysql-test-runs - Results of new strategy
by Pablo Estrada 17 Jul '14

17 Jul '14

Hello everyone, I took the last couple of days working on a new strategy to calculate the relevance of a test. The results are not sufficient by themselves, but I believe they point to an interesting direction. This strategy uses that rate of co-occurrence of events to estimate the relevance of a test, and the events that it uses are the following: - File editions since last run - Test failure in last run The strategy has also two stages: 1. Training stage 2. Executing stage In the training stage, it goes through the available data, and does the following: - If test A failed: - It counts and stores all the files that were edited since the last test_run (the last test_run depends on BRANCH, PLATFORM, and other factors) - If test A failed also in the previous test run, it also counts that In the executing stage, the training algorithm is still applied, but the decision of whether a test runs is based on its relevance, the relevance is calculated as the sum of the following: - The percentage of times a test has failed in two subsequent test_runs, multiplied by whether the test failed in the previous run (if the test didn't fail in the previous run, this quantity is 0) - For each file that was edited since the last test_run, the percentage of times that the test has failed after this file was edited (The explanation is a bit clumsy, I can clear it up if you wish so) The results have not been too bad, nor too good. With a running set of 200 tests, a training phase of 3000 test runs, and an executing stage of 2000 test runs, I have achieved recall of 0.50. It's not too great, nor too bad. Nonetheless, while running tests, I found something interesting: - I removed the first factor of the relevance. I decided to not care about whether a test failed in the previous test run. I was only using the file-change factor. Naturally, the recall decreased, from 0.50 to 0.39 (the decrease was not too big)... and the distribution of failed tests in the priority queue had a good skew towards the front of the queue (so it seems that the files help somewhat, to indicate the likelihood of a failure). I attached this chart. An interesting problem that I encountered was that about 50% of the test_runs don't have any file changes nor test failures, and so the relevance of all tests is zero. Here is where the original strategy (a weighted average of failures) could be useful, so that even if we don't have any information to guess which tests to run, we just go ahead and run the ones that have failed the most, recently. I will work on mixing up both strategies a bit in the next few days, and see what comes of that. By the way, I pushed the code to github. The code is completely different, so may be better to wait until I have new results soon. Regards! Pablo

2 11

[Maria-developers] Java Connector Coding till 5th June
by Puneet Dewan 17 Jul '14

17 Jul '14

Hi Everyone, The work done till 5th June: 1.I started studying the connector immediately after my semester exams got over(5th May). 2. Understood the relationship between different classes. 3.Read the MySql documentation for text protocol and map the commands from documentation onto code.This thing gave a clear idea about how things are moving from client to server. 4.Setup the code on Ubuntu using eclipse and tested against database. Setup the Team branch in launchpad. 5.Debug the test classes to see how different classes are interlinked and understood that MySQLProtocol.java is the heart of our JDBC driver. 6.Then I started reading Client Server Protocol documentation i.e ServerSide prepared statement and try to see how and where I have to code it. 7.Then I saw that there is a class MySqlServerSidePreparedStatement.java, Initially I coded the prepareStatement() ,close(),and execute() in that class seperately. But after (Code review) discussion with Massimo, we donot need to use that class. Instead use MySqlPreparedStatement.java to do the same work.So made changes accordingly. 8.I had pushed some of the code which I have written to launchpad and discussed with Massimo last week. 9.Last week I was struck with Wireshark on windows and ubuntu both. I was unable to see the packets going to database and seeing the response. 10.On Monday the problem was resolved as I was using the wrong interface.I should have used loopback.io(127.0.0.1) instead on ubuntu(thanks to Georg). Unfortunately the thing does not work on windows.Tried a lot many times. 11.Now I am working on the execute method of client server protocol as I am getting exception while using that method.Wireshark is playing a great role in how the packets are flowing. >From now I will do weekly blogging and keep everyone updated. Thanks & Regards Puneet.

2 9

[Maria-developers] Help needed TokuDB build errors in Debian
by Otto Kekäläinen 16 Jul '14

16 Jul '14

Hello Rich! For some strange reason, re-building 5.5.37 in Debian unstable stopped working. Any ideas what might be going on here?: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=753222 Recently a build of 5.5.38 generated a similar error: http://paste.ubuntu.com/7747594/ Are these maybe related? I have no idea how to fix these apart from disabling TokuDB in Debian when uploading 5.5.38. -- Check out our blog at http://seravo.fi/blog and follow @ottokekalainen

3 5

[Maria-developers] [GSoC] Week Eight!
by sriram patil 14 Jul '14

14 Jul '14

Hi all, Following is the blog link about my eighth week into GSoC. http://goo.gl/h2mnbv Suggestions and/or reviews are welcome! Thanks, Sriram

1 0

Re: [Maria-developers] [Commits] Rev 4876: Post-fixes after UNIQUE index support was enabled without uniquenessenforcement. in file:///home/psergey/chroot/saucy-x64/home/psergey/dev2/mysql-5.6-rocksdb-r2/
by Roberto Spadim 11 Jul '14

11 Jul '14

=] nice rocksdb, any devel code to test it? i have some key-value databases to test 2014-07-11 13:21 GMT-03:00 Sergey Petrunya <psergey(a)askmonty.org>: > At > file:///home/psergey/chroot/saucy-x64/home/psergey/dev2/mysql-5.6-rocksdb-r2/ > > ------------------------------------------------------------ > revno: 4876 > revision-id: psergey(a)askmonty.org-20140711162114-051drr4tocqpbx30 > parent: psergey(a)askmonty.org-20140711161708-b50tdmlr60djrror > committer: Sergey Petrunya <psergey(a)askmonty.org> > branch nick: mysql-5.6-rocksdb-r2 > timestamp: Fri 2014-07-11 20:21:14 +0400 > message: > Post-fixes after UNIQUE index support was enabled without > uniquenessenforcement. > - Fix compile warning > - Fix the rocksdb.test/result > === modified file 'mysql-test/r/rocksdb.result' > --- a/mysql-test/r/rocksdb.result 2014-07-03 20:58:25 +0000 > +++ b/mysql-test/r/rocksdb.result 2014-07-11 16:21:14 +0000 > @@ -686,8 +686,8 @@ DROP TABLE t33; > # > # MDEV-4081: RocksDB throws error 122 on an attempt to create a table > with unique index > # > +# Unique indexes can be created, but uniqueness won't be enforced > create table t33 (pk int primary key, u int, unique index(u)) > engine=RocksDB; > -ERROR HY000: Got error 187 'Unique indexes are not supported' from ROCKSDB > # > # MDEV-4077: RocksDB: Wrong result (duplicate row) on select with range > # > > === modified file 'mysql-test/t/rocksdb.test' > --- a/mysql-test/t/rocksdb.test 2014-07-03 20:58:25 +0000 > +++ b/mysql-test/t/rocksdb.test 2014-07-11 16:21:14 +0000 > @@ -610,7 +610,8 @@ DROP TABLE t33; > --echo # > --echo # MDEV-4081: RocksDB throws error 122 on an attempt to create a > table with unique index > --echo # > ---error ER_GET_ERRMSG > +#--error ER_GET_ERRMSG > +--echo # Unique indexes can be created, but uniqueness won't be enforced > create table t33 (pk int primary key, u int, unique index(u)) > engine=RocksDB; > > > > === modified file 'storage/rocksdb/ha_rocksdb.cc' > --- a/storage/rocksdb/ha_rocksdb.cc 2014-07-09 23:46:24 +0000 > +++ b/storage/rocksdb/ha_rocksdb.cc 2014-07-11 16:21:14 +0000 > @@ -1051,7 +1051,7 @@ int ha_rocksdb::create(const char *name, > /* > We only support auto_increment at start of the PRIMARY KEY. > */ > - Field *field; > + //Field *field; > //if ((field= table_arg->next_number_field)) > /* TODO mdcallag: disable this for now to let UNIQUE indexes kind of > work > if ((field= table_arg->found_next_number_field)) > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits > -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

Re: [Maria-developers] apt-get upgrade on buildbot
by Kristian Nielsen 11 Jul '14

11 Jul '14

Otto Kekäläinen <otto(a)seravo.fi> writes: > Hello Kristian! > > Could you do a small favour and run 'apt-get upgrade' and then trigger > manually a rebuild in the virtual machine Debian unstable the buildbot > uses at http://buildbot.askmonty.org/buildbot/builders/debpkg-sid > > Thanks! Hm, I tried this, but I got the below errors. Any ideas what I did wrong? - Kristian. buildbot@debian-sid-amd64:~$ sudo apt-get update Hit http://ftp.us.debian.org sid InRelease Hit http://ftp.us.debian.org sid/main Sources/DiffIndex Hit http://ftp.us.debian.org sid/main amd64 Packages/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-en/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-cs/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-ru/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-nl/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-eo/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-ca/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-pl/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-fi/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-hr/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-fr/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-eu/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-sr/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-hu/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-da/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-ro/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-es/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-pt/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-sv/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-sk/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-ko/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-vi/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-uk/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-km/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-ja/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-de/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-it/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-nb/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-zh/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-id/DiffIndex Get:1 http://ftp.us.debian.org sid/main Translation-ru [423 kB] Get:2 http://ftp.us.debian.org sid/main Translation-pl [575 kB] 100% [2 Translation-pl bzip2 0 B] 66.3 kB/s 0sFalling back to normal index file acquire Ign http://ftp.us.debian.org sid/main 2014-07-09-0854.18.pdiff Get:3 http://ftp.us.debian.org sid/main Translation-fr [731 kB] 100% [3 Translation-fr bzip2 0 B] 90.5 kB/s 0sFalling back to normal index file acquire Ign http://ftp.us.debian.org sid/main 2014-07-09-0854.18.pdiff Get:4 http://ftp.us.debian.org sid/main Translation-da [2,414 kB] Get:5 http://ftp.us.debian.org sid/main Translation-es [323 kB] Get:6 http://ftp.us.debian.org sid/main Translation-pt [444 kB] Get:7 http://ftp.us.debian.org sid/main Translation-sk [421 kB] Get:8 http://ftp.us.debian.org sid/main Translation-uk [776 kB] Get:9 http://ftp.us.debian.org sid/main Translation-ja [786 kB] Get:10 http://ftp.us.debian.org sid/main Translation-de [1,772 kB] Get:11 http://ftp.us.debian.org sid/main Translation-it [3,259 kB] Get:12 http://ftp.us.debian.org sid/main Sources [7,333 kB] Get:13 http://ftp.us.debian.org sid/main amd64 Packages [6,820 kB] Get:14 http://ftp.us.debian.org sid/main [116 kB] Get:15 http://ftp.us.debian.org sid/main [35.0 kB] Fetched 26.2 MB in 1min 31s (288 kB/s) W: Failed to fetch xz:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_source_Sources Hash Sum mismatch W: Failed to fetch xz:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_binary-amd64_Packages Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-ru Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-pl Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-fr Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-da Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-es Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-pt Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-sk Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-uk Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-ja Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-de Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-it Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-fi Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-hu Hash Sum mismatch E: Some index files failed to download. They have been ignored, or old ones used instead. buildbot@debian-sid-amd64:~$ sudo apt-get dist-upgrade Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

2 2

[Maria-developers] [GSoC] Suggestions on adding a new method
by sriram patil 11 Jul '14

11 Jul '14

Hi All, I was thinking of adding two new inline methods in struct LEX as follows inline bool is_create_or_replace() { return (create_info.options & HA_LEX_CREATE_REPLACE); } inline bool is_create_if_not_exists() { return (create_info.options & HA_LEX_CREATE_IF_NOT_EXISTS); } With these, statements like if(lex->create_info.options & HA_LEX_CREATE_REPLACE) will be changed to if(lex->is_create_or_replace()) which definitely is more readable. This kind of statements are used in sql_yacc.yy, sql_parse.cc and many object specific implementation .cc files. Should I make the changes or is it unnecessary? Thanks, Sriram

1 0

Re: [Maria-developers] [JIRA] (MDEV-4937) NEED REVIEW: sql_slave_skip_counter does not work with GTID
by Kristian Nielsen 09 Jul '14

09 Jul '14

"Michael Widenius (JIRA)" <jira(a)mariadb.atlassian.net> writes: > Michael Widenius commented on MDEV-4937: > ---------------------------------------- Thanks for review Monty! I have some comments inline, let me know if you still want me to change something. - Kristian. > revno: 4260 > revision-id: knielsen at knielsen-hq.org-20140620104939-xdvn985cgmwy3odd > committer: Kristian Nielsen <knielsen at knielsen-hq.org> > timestamp: Fri 2014-06-20 12:49:39 +0200 > message: > MDEV-4937: sql_slave_skip_counter does not work with GTID > === modified file 'sql/slave.cc' > --- a/sql/slave.cc 2014-06-19 12:50:43 +0000 > +++ b/sql/slave.cc 2014-06-20 10:49:39 +0000 > @@ -3521,9 +3521,6 @@ static int exec_relay_log_event(THD* thd > > if (opt_gtid_ignore_duplicates) > { > - serial_rgi->current_gtid.domain_id= gev->domain_id; > - serial_rgi->current_gtid.server_id= gev->server_id; > - serial_rgi->current_gtid.seq_no= gev->seq_no; > int res= rpl_global_gtid_slave_state.check_duplicate_gtid > (&serial_rgi->current_gtid, serial_rgi); > if (res < 0) > What was the reason for removing the above assignments? > Was it a bug in the old code ? Not really a bug, however, just before this code we have: if (event_group_new_gtid(serial_rgi, gev)) and event_group_new_gtid() does the exact same assignments, so the ones I removed are completely redundant. I just happened to notice this redundancy, so I removed them... > (I have to ask as I am not 100% of what the old code did). > @@ -4594,16 +4598,27 @@ log '%s' at position %s, relay log '%s' > > if (saved_skip && rli->slave_skip_counter == 0) > { > + String tmp; > + if (mi->using_gtid != Master_info::USE_GTID_NO) > + { > + tmp.append(STRING_WITH_LEN(", GTID '")); > + rpl_append_gtid_state(&tmp, false); > + tmp.append(STRING_WITH_LEN("'; ")); > + } > > {noformat} > > If you want to avoid checking if tmp.append() fails, you could > create tmp with a static buffer. > > char tmp_buff[MAX_GTID_LENGTH+10]; > String tmp(tmp_buff, sizeof(tmp_buff), &my_charset_bin); > tmp.length(0); Right, hm. It does not really feel right to enforce a limit on the length; if the user uses a longer GTID position (which would be rather unusual), it might be confusing to have it silently cut. If the append() should fail, well, then it _will_ be cut, but then it's just an informational message, and that kind of out-of-memory is probably unlikely (and the error log will likely be filled up with other complaints about out-of-memory). But let me know if you still think I should change it. (If so - can you think of a place where MAX_GTID_LENGTH can be defined, that would be accessible to both client/mysqldump.c and sql/slave.cc? I could spot only my_global.h, which seems a bit too generic to contain GTID-related stuff...?) > if (saved_skip && rli->slave_skip_counter == 0) > { > String tmp; > saved_skip= 0; > + saved_skip_gtid_pos.free(); > } > If you free saved_skip_gtid_pos, it good to also free tmp! Right ... but it should be freed automatically anyway, when tmp goes out of scope, right? > === modified file 'sql/sys_vars.cc' > --- a/sql/sys_vars.cc 2014-06-09 18:00:23 +0000 > +++ b/sql/sys_vars.cc 2014-06-20 10:49:39 +0000 > @@ -4287,11 +4287,6 @@ bool update_multi_source_variable(sys_va > > static bool update_slave_skip_counter(sys_var *self, THD *thd, Master_info *mi) > { > - if (mi->using_gtid != Master_info::USE_GTID_NO) > - { > - my_error(ER_SLAVE_SKIP_NOT_IN_GTID, MYF(0)); > - return true; > - } > > {noformat} > > Shouldn't you also remove the error ER_SLAVE_SKIP_NOT_IN_GTID from the > error file? I think we cannot remove error codes from a GA release once introduced, right? So I left it, to not shift other error numbers. But let me know if I'm wrong, and I'll remove it. - Kristian.

1 0

Re: [Maria-developers] [JIRA] (MDEV-6336) NEED REVIEW: mysqldump --master-data does not work with GTID setups
by Kristian Nielsen 09 Jul '14

09 Jul '14

"Michael Widenius (JIRA)" <jira(a)mariadb.atlassian.net> writes: > Here is the review: > > ------------------------------------------------------------ > revno: 4245 > revision-id: knielsen at knielsen-hq.org-20140616093803-gub60pda6t77ch3k > committer: knielsen at knielsen-hq.org > timestamp: Mon 2014-06-16 11:38:03 +0200 > message: > MDEV-6336: mysqldump --master-data does not work with GTID setups > MDEV-6344: mysqldump issues FLUSH TABLES, which gets written into binlog and replicated Thanks for review Monty! I fixed all of your suggestions. - Kristian.

1 0

Re: [Maria-developers] [Commits] Rev 4281: MDEV-5262, MDEV-5914, MDEV-5941, MDEV-6020: Deadlocks during parallel replication causing replication to fail. in http://bazaar.launchpad.net/~maria-captains/maria/10.0
by Kristian Nielsen 09 Jul '14

09 Jul '14

Jan Lindström <jan.lindstrom(a)skysql.com> writes: > I have mostly InnoDB style comments, but some questions below: Thanks for comments! Sorry about style violations, and thanks for pointing then out. I'll try to fix all of it. Answers to other questions below: > Why you need to define these, why #include "mysql/plugin.h" file is not > enough ? > > +extern "C" void thd_report_wait_for(const MYSQL_THD thd, MYSQL_THD > other_thd); > +extern "C" int thd_need_wait_for(const MYSQL_THD thd); > +extern "C" > +int thd_need_ordering_with(const MYSQL_THD thd, const MYSQL_THD other_thd); See the discussions here: https://lists.launchpad.net/maria-developers/msg07494.html The conclusion with Serg was that doing a proper public API in 10.0 was too big a change. We will do it in 10.1 (MDEV-6429). plugin.h should not be modified in a GA version and with an API that has not been thought through wrt. all engines. Also, the API should be a service, not in plugin.h. > Why you need to introduce a new global structure for this, can't this be > part > of some already existing system structure e.g. lock_sys and why you do not > use ut_list ? > > +/* Buffer to collect THDs to report waits for. */ > +struct thd_wait_reports { > + struct thd_wait_reports *next; > + ulint used; > + trx_t *waitees[64]; > +}; So the idea here is to collect a bunch of trx pointers during the graph traversal, and pass each of them to thd_report_wait_for(). I wanted to pass them after completing the graph traversal to avoid issues if thd_report_wait_for() needs to kill one of them; this goes to lock_cancel_waiting_and_release() eventually which I assume might modify the graph being traversed. This is time-critical code (it runs under global lock_sys->mutex), so it is important to avoid memory allocations. So I use an array allocated on the stack, this is really short-lived data, it only lives for the duration of lock_deadlock_check_and_resolve(). In all normal cases the single array should be sufficient, but the code needs to handle the overflow also, of course. So in case of overflow, I malloc further arrays and link them into the first. I suppose another solution would be to abort in case of overflow and roll back the transaction like in case of ctx->too_deep==true, however I don't like to introduce such arbitrary limits when it's not necessary. I looked into using ib_list, but this seems to always do mallocs, which I want to avoid. I also looked at ut0vec and dyn0dyn, but they also seem to always use malloc and thus do not seem appropriate. So I did not find anything existing I could use, unfortunately, but then I know little of the InnoDB code. Do you have a suggestion for an existing data structure to use that can take an inital buffer on the stack and malloc extra memory only if needed? Thanks for comments! - Kristian.

1 0

Re: [Maria-developers] [Commits] Rev 4275: MDEV-5867 ALTER TABLE t1 ENGINE=InnoDB keeps bad options when t1 ENGINE is CONNECT in lp:~maria-captains/maria/10.0
by Sergey Vojtovich 08 Jul '14

08 Jul '14

Hi Sergei, I believe options handling is way too confusing in general: - CREATE TABLE warns/errors about unknown options - ALTER TABLE ... ENGINE doesn't - ALTER TABLE ... UNKNOWN_OPTION=1 does - CREATE TABLE ... LIKE doesn't - different engines may assign different meaning to the same option - IGNORE_BAD_TABLE_OPTIONS doesn't actually _ignore_ _bad_ options but rather _accepts_ _unkown_ options But the question is if we should show unknown options by default. I have no opinion: both cases are equally wrong to me. On the one hand we get unusable CREATE TABLE statement, on the other hand hidden metadata which may affect further statements. I'm fine with hidding unknown options that were accepted due to IGNORE_BAD_TABLE_OPTIONS, but hidding things that were accepted by fully successful ALTER TABLE statement sounds confusing. Regards, Sergey On Sat, Jul 05, 2014 at 11:42:25PM +0200, Sergei Golubchik wrote: > At lp:~maria-captains/maria/10.0 > > ------------------------------------------------------------ > revno: 4275 > revision-id: sergii(a)pisem.net-20140705214225-p0k45asdaq27xocr > parent: sergii(a)pisem.net-20140705214216-npqtc7tz854xqjxo > fixes bug: https://mariadb.atlassian.net/browse/MDEV-5867 > committer: Sergei Golubchik <sergii(a)pisem.net> > branch nick: 10.0 > timestamp: Sat 2014-07-05 23:42:25 +0200 > message: > MDEV-5867 ALTER TABLE t1 ENGINE=InnoDB keeps bad options when t1 ENGINE is CONNECT > > don't print unknown options in SHOW CREATE TABLE unless IGNORE_BAD_TABLE_OPTIONS is used > === added file 'mysql-test/r/table_options-5867.result' > --- a/mysql-test/r/table_options-5867.result 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/r/table_options-5867.result 2014-07-05 21:42:25 +0000 > @@ -0,0 +1,37 @@ > +install soname 'ha_example'; > +set sql_mode='ignore_bad_table_options'; > +create table t1 ( > +a int complex='c,f,f,f' invalid=3 > +) engine=example ull=10000 str='dskj' one_or_two='one' yesno=0 > +foobar=barfoo; > +Warnings: > +Warning 1911 Unknown option 'invalid' > +Warning 1911 Unknown option 'foobar' > +create table t2 (a int, key (a) some_option=2014); > +Warnings: > +Warning 1911 Unknown option 'some_option' > +show create table t1; > +Table Create Table > +t1 CREATE TABLE `t1` ( > + `a` int(11) DEFAULT NULL `complex`='c,f,f,f' `invalid`=3 > +) ENGINE=EXAMPLE DEFAULT CHARSET=latin1 `ull`=10000 `str`='dskj' `one_or_two`='one' `yesno`=0 `foobar`=barfoo `VAROPT`='5' > +show create table t2; > +Table Create Table > +t2 CREATE TABLE `t2` ( > + `a` int(11) DEFAULT NULL, > + KEY `a` (`a`) `some_option`=2014 > +) ENGINE=MyISAM DEFAULT CHARSET=latin1 > +set sql_mode=''; > +show create table t1; > +Table Create Table > +t1 CREATE TABLE `t1` ( > + `a` int(11) DEFAULT NULL `complex`='c,f,f,f' > +) ENGINE=EXAMPLE DEFAULT CHARSET=latin1 `ull`=10000 `str`='dskj' `one_or_two`='one' `yesno`=0 `VAROPT`='5' > +show create table t2; > +Table Create Table > +t2 CREATE TABLE `t2` ( > + `a` int(11) DEFAULT NULL, > + KEY `a` (`a`) > +) ENGINE=MyISAM DEFAULT CHARSET=latin1 > +drop table t1, t2; > +uninstall soname 'ha_example'; > > === modified file 'mysql-test/suite/rpl/r/rpl_table_options.result' > --- a/mysql-test/suite/rpl/r/rpl_table_options.result 2013-09-14 01:09:36 +0000 > +++ b/mysql-test/suite/rpl/r/rpl_table_options.result 2014-07-05 21:42:25 +0000 > @@ -12,6 +12,12 @@ show create table t1; > Table Create Table > t1 CREATE TABLE `t1` ( > `a` int(11) NOT NULL > +) ENGINE=MyISAM DEFAULT CHARSET=latin1 > +set sql_mode=ignore_bad_table_options; > +show create table t1; > +Table Create Table > +t1 CREATE TABLE `t1` ( > + `a` int(11) NOT NULL > ) ENGINE=MyISAM DEFAULT CHARSET=latin1 `ull`=12340 > drop table t1; > set storage_engine=default; > > === modified file 'mysql-test/suite/rpl/t/rpl_table_options.test' > --- a/mysql-test/suite/rpl/t/rpl_table_options.test 2011-10-19 19:45:18 +0000 > +++ b/mysql-test/suite/rpl/t/rpl_table_options.test 2014-07-05 21:42:25 +0000 > @@ -23,6 +23,8 @@ show create table t1; > sync_slave_with_master; > connection slave; > show create table t1; > +set sql_mode=ignore_bad_table_options; > +show create table t1; > > connection master; > drop table t1; > > === added file 'mysql-test/t/table_options-5867.test' > --- a/mysql-test/t/table_options-5867.test 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/t/table_options-5867.test 2014-07-05 21:42:25 +0000 > @@ -0,0 +1,30 @@ > +# > +# MDEV-5867 ALTER TABLE t1 ENGINE=InnoDB keeps bad options when t1 ENGINE is CONNECT > +# > +# verify that SHOW CREATE TABLE hides unknown options when IGNORE_BAD_TABLE_OPTIONS is not set > + > +--source include/have_example_plugin.inc > +--source include/not_embedded.inc > + > +install soname 'ha_example'; > + > +set sql_mode='ignore_bad_table_options'; > +create table t1 ( > + a int complex='c,f,f,f' invalid=3 > +) engine=example ull=10000 str='dskj' one_or_two='one' yesno=0 > + foobar=barfoo; > + > +create table t2 (a int, key (a) some_option=2014); > + > +show create table t1; > +show create table t2; > + > +set sql_mode=''; > + > +show create table t1; > +show create table t2; > + > +drop table t1, t2; > + > +uninstall soname 'ha_example'; > + > > === modified file 'sql/create_options.cc' > --- a/sql/create_options.cc 2014-03-26 08:33:03 +0000 > +++ b/sql/create_options.cc 2014-07-05 21:42:25 +0000 > @@ -775,3 +775,20 @@ engine_option_value *merge_engine_table_ > &first, &end); > DBUG_RETURN(first); > } > + > +bool is_engine_option_known(engine_option_value *opt, > + ha_create_table_option *rules) > +{ > + if (!rules) > + return false; > + > + for (; rules->name; rules++) > + { > + if (!my_strnncoll(system_charset_info, > + (uchar*)rules->name, rules->name_length, > + (uchar*)opt->name.str, opt->name.length)) > + return true; > + } > + return false; > +} > + > > === modified file 'sql/create_options.h' > --- a/sql/create_options.h 2013-11-20 11:05:39 +0000 > +++ b/sql/create_options.h 2014-07-05 21:42:25 +0000 > @@ -99,4 +99,6 @@ uchar *engine_table_options_frm_image(uc > > bool engine_options_differ(void *old_struct, void *new_struct, > ha_create_table_option *rules); > +bool is_engine_option_known(engine_option_value *opt, > + ha_create_table_option *rules); > #endif > > === modified file 'sql/sql_show.cc' > --- a/sql/sql_show.cc 2014-07-05 21:42:16 +0000 > +++ b/sql/sql_show.cc 2014-07-05 21:42:25 +0000 > @@ -1519,13 +1519,20 @@ static bool get_field_default_value(THD > @param thd thread handler > @param packet string to append > @param opt list of options > + @param check_options only print known options > + @param rules list of known options > */ > > static void append_create_options(THD *thd, String *packet, > - engine_option_value *opt) > + engine_option_value *opt, > + bool check_options, > + ha_create_table_option *rules) > { > for(; opt; opt= opt->next) > { > + if (check_options && !is_engine_option_known(opt, rules)) > + continue; > + > DBUG_ASSERT(opt->value.str); > packet->append(' '); > append_identifier(thd, packet, opt->name.str, opt->name.length); > @@ -1584,6 +1591,8 @@ int show_create_table(THD *thd, TABLE_LI > MODE_MAXDB | MODE_ANSI); > bool limited_mysql_mode= sql_mode & (MODE_NO_FIELD_OPTIONS | MODE_MYSQL323 | > MODE_MYSQL40); > + bool check_options= !(sql_mode & MODE_IGNORE_BAD_TABLE_OPTIONS) && > + !create_info_arg; > handlerton *hton; > my_bitmap_map *old_map; > int error= 0; > @@ -1733,7 +1742,8 @@ int show_create_table(THD *thd, TABLE_LI > packet->append(STRING_WITH_LEN(" COMMENT ")); > append_unescaped(packet, field->comment.str, field->comment.length); > } > - append_create_options(thd, packet, field->option_list); > + append_create_options(thd, packet, field->option_list, check_options, > + hton->field_options); > } > > key_info= table->key_info; > @@ -1800,7 +1810,8 @@ int show_create_table(THD *thd, TABLE_LI > append_identifier(thd, packet, parser_name->str, parser_name->length); > packet->append(STRING_WITH_LEN(" */ ")); > } > - append_create_options(thd, packet, key_info->option_list); > + append_create_options(thd, packet, key_info->option_list, check_options, > + hton->index_options); > } > > /* > @@ -1953,7 +1964,8 @@ int show_create_table(THD *thd, TABLE_LI > packet->append(STRING_WITH_LEN(" CONNECTION=")); > append_unescaped(packet, share->connect_string.str, share->connect_string.length); > } > - append_create_options(thd, packet, share->option_list); > + append_create_options(thd, packet, share->option_list, check_options, > + hton->table_options); > append_directory(thd, packet, "DATA", create_info.data_file_name); > append_directory(thd, packet, "INDEX", create_info.index_file_name); > } > @@ -5130,7 +5142,7 @@ static int get_schema_tables_record(THD > str.qs_append(STRING_WITH_LEN(" transactional=")); > str.qs_append(ha_choice_values[(uint) share->transactional]); > } > - append_create_options(thd, &str, share->option_list); > + append_create_options(thd, &str, share->option_list, false, 0); > > if (str.length()) > table->field[19]->store(str.ptr()+1, str.length()-1, cs); > > === modified file 'storage/connect/mysql-test/connect/r/alter.result' > --- a/storage/connect/mysql-test/connect/r/alter.result 2014-03-23 14:50:39 +0000 > +++ b/storage/connect/mysql-test/connect/r/alter.result 2014-07-05 21:42:25 +0000 > @@ -218,13 +218,22 @@ Three 3 > # Changing to another engine is Ok > # However, the data file is not deleted. > # > -ALTER TABLE t1 ENGINE=MARIA; > +ALTER TABLE t1 ENGINE=ARIA; > +SHOW CREATE TABLE t1; > +Table Create Table > +t1 CREATE TABLE `t1` ( > + `d` char(10) NOT NULL, > + `c` int(11) NOT NULL > +) ENGINE=Aria DEFAULT CHARSET=latin1 PAGE_CHECKSUM=1 > +set @old_sql_mode=@@sql_mode; > +set sql_mode=ignore_bad_table_options; > SHOW CREATE TABLE t1; > Table Create Table > t1 CREATE TABLE `t1` ( > `d` char(10) NOT NULL `FLAG`=11, > `c` int(11) NOT NULL `FLAG`=0 > ) ENGINE=Aria DEFAULT CHARSET=latin1 PAGE_CHECKSUM=1 `TABLE_TYPE`=fix `FILE_NAME`='tf1.txt' `ENDING`=1 > +set sql_mode=@old_sql_mode; > SELECT * from t1; > d c > One 1 > > === modified file 'storage/connect/mysql-test/connect/t/alter.test' > --- a/storage/connect/mysql-test/connect/t/alter.test 2014-03-23 14:50:39 +0000 > +++ b/storage/connect/mysql-test/connect/t/alter.test 2014-07-05 21:42:25 +0000 > @@ -105,8 +105,12 @@ SELECT * FROM t1; > --echo # Changing to another engine is Ok > --echo # However, the data file is not deleted. > --echo # > -ALTER TABLE t1 ENGINE=MARIA; > +ALTER TABLE t1 ENGINE=ARIA; > SHOW CREATE TABLE t1; > +set @old_sql_mode=@@sql_mode; > +set sql_mode=ignore_bad_table_options; > +SHOW CREATE TABLE t1; > +set sql_mode=@old_sql_mode; > SELECT * from t1; > SELECT * from t2; > > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

4 7

[Maria-developers] Java Connector coding till 6th July
by Puneet Dewan 08 Jul '14

08 Jul '14

Hi Everyone, Sorry for the late mail. Last week I had code review with Massimo. I wrote the test cases for transactions. Then I clean the code written in MySQLProtocol.java for executePreparedStatement method. Then added new methods like writeBinaryTo(for BinaryProtocol) in IntParameter and will write similar methods in other classes. In the coming week: *a)*I will be making use of MYSQLServerSidePreparedStatement.java for performing the all the operations.Currently they are embedded in MYSQLPreparedStatement.java *b)*I have done the analysis of the above as to what classes will be changed. *c)*Also I will be modifying the execute method for Serverside prepared statements for sending the *new params bound flag* properly. Currently it is always going as 0x01 but will modify it to send 0x00 if same statement gets executed again I will have code review with Massimo on thursday.. Your suggestions are welcome. Regards Puneet.

2 2

Re: [Maria-developers] comments to the plugin.h
by Kristian Nielsen 08 Jul '14

08 Jul '14

Sergei Golubchik <vuvova(a)gmail.com> writes: > This is what I've written in the patch: I've tried to answer all points in detail, also to help myself better understand all the issues: >> + The storage engine can report such conflicting locks using this call. This >> + will allow parallel replication to detect such conflicts and resolve the >> + deadlock (by killing the second transaction to release the locks that the >> + first is waiting for, and then later re-try the second killed transaction). > > What will happen if the engine doesn't? Deadlocks will timeout. > Which is annoying, but it only affects the performance, not the > correctness. No, unfortunately that is not so. Let's say we have T1 and T2. T1 goes to wait on a row lock held by T2. Then T2 completes its prepare phase, it goes to the group commit queue and waits for T1 to complete and commit for both of them. T1 gets lock wait timeout, so T1 will roll back and try again. But T1 will just go to wait on the same row lock again that is held by T2. So T1 will retry over and over until slave_transaction_retries is exceeded, at which point replication will abort with an error. There is no lock wait timeout when T2 waits in the group commit queue. > But can it cause the first transaction to be killed, not > the second? Then it's worse - it'll violate the strict commit ordering. No, this can never happen. Commit order is enforced by the wait_for_commit facility. T2 will not be allowed to commit before T1. If T1 is killed and retried, T2 will continue waiting for T1 to commit the second time. And if T1 fails, it will call wakeup_subsequent_commits() with an error, so that T1 can abort its commit. > And, again, the engine needs to do it for every lock wait. Even while most of > them won't cause any deadlocks. Right. This is only needed for parallel replication worker threads. That is why I introduced thd_need_wait_for(). The storage engine can call this once for a transaction or even for a THD, and save the result. And then it won't need to call thd_report_wait_for() except for the parallel replication worker threads. And for those, lock waits should be rare. >> + The storage engine should not report false positives. That is, it should not >> + report any lock waits that do not actually require one transaction to wait >> + for the other. Nor should it report waits for locks that will be released >> + before the commit of the other transactions. > > What will happen if the engine reports false positives? Lock waist > that don't require one transaction to wait for another. What if it > report waits for locks that will be released before commit of the other > transaction? If it reports that T1 will wait for T2, then we will kill T2 and retry it. This will cause unnecessary work if there is no actual deadlock, so mainly a performance issue. A deeper problem is caused by the async kill processing. If T1 is reported as waiting for T2, we will kill T2 in the background thread. But if there is no such wait, or if the wait is resolved due to T2 releasing locks before commit, then T1 and T2 could both complete, and their worker threads move on to new transactions. Then the background thread will be trying to kill a T2 which no longer exists, which will access incorrect rpl_group_info objects. This is the part of the patch I am most unhappy about :-/ But we started to discuss on IRC to possibly avoid the async kill, instead making it possible to call THD::awake() directly inside thd_report_wait_for(). This would be good, as it would solve the deeper problem. In this case, a false positive would then cause only performance issues, I think, though I should think the problem through carefully to be sure. >> +void thd_report_wait_for(const MYSQL_THD thd, MYSQL_THD other_thd); > > You kill the second thread in a background. > What if you convey the ordering to the engine and let it know about a > deadlock? Like, if you return 1 from thd_report_wait_for() > it means a deadlock and the engine will kill the transaction > just normally as for any normal deadlock case. The problem is that T2 could be executing anywhere in the server, including server code or another storage engine, where InnoDB does not really have any possibility to kill it. This is different from a normal InnoDB deadlock, which only happens if both T1 and T2 are doing an actual row lock wait inside the storage engine itself. But in this case, only T1 is waiting, T2 is continuing whatever processing it is doing, eventually it will end up in the group commit queue in binlog or in wait_for_prior_commit(), and wait for T1 to commit first. >> +int thd_need_wait_for(const MYSQL_THD thd); > > I'd suggest to remove it. If you think it's useful, you can call it from > your thd_report_wait_for() and spare storage engine the knowledge of yet > another API function. See discussion above. The intention is to avoid lots of calls to thd_report_wait_for() from the storage engine for non-replication threads. So calling it from inside thd_report_wait_for() makes no sense. I don't understand the benefits of reducing the functionality available in the API when that functionality is completely optional? But on the other hand, I do not feel strongly about this function. I do not even use it in the patch, as InnoDB does a rather expensive graph traversal of all waiters anyway in its deadlock detector, so it can be just omitted. I included this function because without it, there seems no way for a storage engine to avoid introducing O(N**2) overhead for N transactions contending for a row lock. That seemed unacceptable to me. >> +int thd_need_ordering_with(const MYSQL_THD thd, const MYSQL_THD other_thd); > > Hmm, really? Couldn't you simply disable gap locks in some other already > existing way? Like decrease the transaction isolation level or > tell the engine that it's RBR? No, that is not safe. Suppose we have two transactions: T1: INSERT INTO t1 VALUES (1); T2: INSERT INTO t2 SELECT * FROM t1; T1 and T2 are not run in parallel, because they cannot group commit together on the master. However, on the slave, as soon as T1 _starts_ to commit, T2 is allowed to start. But T2 cannot see the changes done by T1 until T1 _completes_ its commit. Thus it is critical that gap locks are disabled _only_ among transactions within the same group commit, not between different group commits. In fact there is a bug in the patch you got regarding this, as it did not check the commit_id before relaxing gap locks. I currently have the following extra patch in my tree to fix this: === modified file 'sql/sql_class.cc' --- sql/sql_class.cc 2014-06-10 08:13:15 +0000 +++ sql/sql_class.cc 2014-06-27 11:47:53 +0000 @@ -4277,6 +4277,8 @@ thd_need_ordering_with(const MYSQL_THD t return 1; if (rgi->current_gtid.domain_id != other_rgi->current_gtid.domain_id) return 1; + if (rgi->commit_id != other_rgi->commit_id) + return 1; /* These two threads are doing parallel replication within the same replication domain. Their commit order is already fixed, so we do not need On the other hand, if we were to omit this part of the patch completely and just keep the gap locks always, then things would still work, because thd_report_wait_for() would detect a deadlock and do a retry to resolve it. But we would get extra rollbacks. Maybe such extra rollbacks would be rare, though. >> +int thd_deadlock_victim_preference(const MYSQL_THD thd1, const MYSQL_THD thd2); >> + > > Okay, I see why this one is needed. > > On the other hand... > It seems like it's the only function that you realy need. > Because > > thd_report_wait_for(const MYSQL_THD thd, MYSQL_THD other_thd); > > can be replaced with > > if (thd_deadlock_victim_preference(thd, other_thd) < 0) goto deadlock; > > And > > thd_need_wait_for(const MYSQL_THD thd); > > can be removed (replaced with noop). And > > thd_need_ordering_with(const MYSQL_THD thd, const MYSQL_THD other_thd); > > can be replaced with > > thd_deadlock_victim_preference(thd, other_thd) != 0 I don't understand the purpose of using the same function for these? They do completely different things, using a single function will only cause confusion, it seems to me: 1. thd_report_wait_for() is used to allow the _replication code_ to detect a deadlock related to ordering. 2. thd_deadlock_victim_preference() is only used when the _storage engine_ detects a deadlock, replication related or not. 3. thd_need_ordering_with() is used by the storage engine to decide on lock wait semantics. > Although in that case thd_deadlock_victim_preference() should better be > renamed, because it won't be a generic "victim preference", > but specifically victim preference, depending on the fixed transaction > commit ordering. A better name could be, for example, > > thd_get_commit_order(thd, other_thd) > -1 - thd first > 0 - unspecified > +1 - other_thd first How can the storage engine know the difference between T1 and T2 being in the same group commit, or in different group commits? How can the storage engine inquire about need for gap locks, without accidentally reporting a lock wait to the upper layer? > I think it'll be much simpler with only one function. Why? I think it's quite the opposite... > And also it becomes clear what happens if the engine doesn't use it. > This function is used to enquire about the commit order, if it is > not used - the commit order cannot be guaranteed. No, that is not the case. On the contrary - the storage engine is free not to use this, commit order will be guaranteed in any case by wait_for_commit(). Hm. So my idea was to make the new API simple and generic, the upper layer would handle most of the detail about commit ordering or whatever. The storage engine only needs to think about simple stuff like lock waits or deadlock victims, both of which it likely already needs to deal with anyway. But if I understand correctly, another idea could be to make the new API specific just to this particular problem of enforced commit order in parallel replication. So the storage engine would be able to know of such ordering, and should then handle any necessary actions on this by itself? > And you won't need background killing for slave threads The need for background killing comes because it does not work to call THD::awake() from the context of where the InnoDB deadlock detector calls thd_report_wait_for(). The problem is that innobase_kill_query() does a recursive mutex lock on lock_sys->mutex. We discussed on IRC to fix this so background kill is not necessary. One way would be to document that hton->kill_query must be safe to call from within thd_report_wait_for(). Maybe that is the simplest way. It should not be hard to fix the InnoDB code to make this work. For example, InnoDB could just set a flag in the transaction that it already has lock_sys->mutex held, thus avoiding the recursive mutex lock. You also had a different suggestion: > here's another idea. To have thd_get_commit_order() as I thought originally > *and* allow innodb to kill other threads, that is, add a function for > plugins to invoke THD::awake. This function should not call into the engine > that invoked it. That is, supposedly the engine knows how to kill threads > stuck in that engine, to kill other threads it should use this new helper. > In that case innodb can detect the deadlock and can kill the other thread > and it won't break on lock mutex > The underlying logic of this killer helper would be "an engine knows how to > kill a transaction waiting in this engine, so we wouldn't bother doing > that. but to kill a transaction waiting elsewhere the engine needs help" It seems better to me if all the details of enforcing commit order and killing worker threads in case of deadlocks could be kept inside the replication code? Otherwise every storage engine would potentially need to re-implement this? I think that is what I tried to do with my proposed storage engine API, to cleanly separate the storage engine's responsibilities, which is about row-level locks and internal deadlock victims, from the replication codes responsibilities. Wouldn't it be simpler to do it like this? 1. InnoDB deadlock detector reports lock waits to whatever function serves the thd_report_wait_for() purpose. 2. We document that thd_report_wait_for() may need to call hton->kill_query(). 3. We fix InnoDB to handle (2) correctly (assuming this is possible to do in a simple way). > And anyway, this absolutely must be in a service, not in the global plugin.h. Why? My understanding was that we do not support ABI stability for storage engines. (And I think we should absolutely not provide such support for storage engines. Storage engines are all about performance and flexibility - ABI stability will make this harder to provide without sufficient other benefits to justify this. ABI seems mainly suited for eg. authentication plugins). > See? In this patch you've removed a function from the global plugin API Right, the thd_rpl_is_parallel(). That should have never be allowed into the code in the first place, it was a temporary hack only. :-( > header, strictly speaking, you must increase the API version - the major > number of - making all older plugins incompatible with the new server. This > is a bit extreme, taking into account that no plugin actually uses > thd_rpl_is_parallel() > To avoid this, please only add new APIs as services. > See libservices/HOWTO I can do that, if you want. It just seems a needless complication to me... Are there any performance penalties for using a service over a normal function call (for built-in storage engines)? I seem to recall there is not? - Kristian.

2 6

Re: [Maria-developers] [Commits] Rev 4274: small cleanup of the SHOW CREATE TABLE code in lp:~maria-captains/maria/10.0
by Sergey Vojtovich 08 Jul '14

08 Jul '14

Hi Sergei, looks good. One question inline. On Sat, Jul 05, 2014 at 11:42:16PM +0200, Sergei Golubchik wrote: > At lp:~maria-captains/maria/10.0 > > ------------------------------------------------------------ > revno: 4274 > revision-id: sergii(a)pisem.net-20140705214216-npqtc7tz854xqjxo > parent: sergii(a)pisem.net-20140704190257-1z1inye8jd7k7umc > committer: Sergei Golubchik <sergii(a)pisem.net> > branch nick: 10.0 > timestamp: Sat 2014-07-05 23:42:16 +0200 > message: > small cleanup of the SHOW CREATE TABLE code ...skip... > === modified file 'sql/sql_show.cc' > --- a/sql/sql_show.cc 2014-06-05 22:07:27 +0000 > +++ b/sql/sql_show.cc 2014-07-05 21:42:16 +0000 > @@ -1578,27 +1577,32 @@ int store_create_info(THD *thd, TABLE_LI > handler *file= table->file; > TABLE_SHARE *share= table->s; > HA_CREATE_INFO create_info; > -#ifdef WITH_PARTITION_STORAGE_ENGINE > bool show_table_options= FALSE; > -#endif /* WITH_PARTITION_STORAGE_ENGINE */ Why did you remove ifdef-s here? This variable seem to be unused without partitioning. ...skip... Regards, Sergey

2 1

[Maria-developers] doubt about commit and myisam/aria engine
by Roberto Spadim 07 Jul '14

07 Jul '14

hi guys, i'm with a doubt... when a aria or myisam file is fsync to disk? when i execute commit? when i execute a autocommit? -- Roberto Spadim SPAEmpresarial

1 0

[Maria-developers] [GSoC] Week Seven!
by sriram patil 07 Jul '14

07 Jul '14

Hi all, Following is the blog link about my seventh week into GSoC. http://goo.gl/jBeubY Suggestions and/or reviews are welcome! Thanks, Sriram

1 0

[Maria-developers] comments on parallel applying
by Jonas Oreland 04 Jul '14

04 Jul '14

hi there, disclaimer: i haven't followed all turns and details on this task so sorry if my comments are outdated (or generally uninteresting). --- i've been glancing on the review comments on the parallel applying, and have some comments to make. i think it will be well invested time to add infrastructure to be able to restrict when parallel applying will be performed. typically i think it would be nice to - only allow parallel transaction in same engine - decrease parallelism when performing DDL ...(more of these can probably be added) restrictions that will make it easier to reason about correctness and that will not affect performance for 99% of the cases (i think :-) if adding such restrictions was an option, the resolving review comments on weird corner cases would be much easier. --- from what I can tell, there are currently no such restrictions, and no infrastructure to add them either. i realize that it might not be trivial to introduce such restrictions, but personally i think it will be worth the effort to get a solid solution in place. <quick thoughts on implementation> for row-based replication this seems quite "easy". for statement-based replication i image that you would have to add hooks into the "real" code after parsing has been performed, but before the actual execution is started (and yes, i know that there is sometimes a blurry line here) </thoughts> --- /Jonas

3 4

[Maria-developers] Review request for MDEV-6020, MDEV-5914, MDEV-5941, and MDEV-5262
by Kristian Nielsen 04 Jul '14

04 Jul '14

Hi Serg, We discussed on IRC a review of the $subj bugs related to deadlocks in parallel replication and handling of automatic retry of transactions in case of deadlock. The patches are in my tree: lp:~maria-captains/maria/10.0-knielsen The complete diff can be obtained eg. with: bzr diff -rrevid:psergey@askmonty.org-20140429143317-tkmxrzu6y36hpi7t..revid:knielsen@knielsen-hq.org-20140610081315-o5nx3u32uvqaai7q Let me give some background for these bugs: When two transactions T1 and T2 group-commit on the master, we assume that they have no lock conflicts (otherwise one would have had to wait for the other to commit). So we run them in parallel on the slave. But before we let T2 commit, we wait until T1 has committed first, so that we get the same binlog order on the slave as we had on the master. But if T1 and T2 somehow _do_ get a lock conflict on the slave, and T1 ends up waiting for some lock held by T2, we get a deadlock. This causes the replication to hang (the wait of T2 for T1 to commit is not part of InnoDB deadlock detection), which is quite bad. Elena's testing of the parallel replication in fact uncovered a number of cases where such deadlock issues occur: 1. MDEV-5914. It turns out that InnoDB gap locks can work in a non-symmetric way. If T1 is an INSERT, and T2 is a DELETE, then T1 can run first and not block T2. But if T2 runs first, then it may set a gap lock that blocks T1. Thus, depending on exact execution order, we can get T1 and T2 to group commit on the master, but T2 block T1 on the slave and cause a deadlock. 2. MDEV-5941. If eg. an UPDATE uses different execution plans on the master and slave, it might use index scan on the master, but table scan on the slave. Thus it may take more locks on the slave than on the master, causing it to conflict with another transaction that is run in parallel and cause a deadlock. 3. MDEV-6020. Here, there is a multi-value insert into multiple partitions of a table with auto-increment key. It appears that in this case InnoDB takes an auto-increment table lock on each partition. Since these locks are taken in the order of the inserted values, they can be taken in different order between different INSERT statements, and can cause a deadlock. Since these locks are released during the statement execution (before the commit), such deadlock can occur on the slave even though it did not occur on the master. (Note that this deadlock _is_ detected by InnoDB, and is not related to the commit order on the parallel slave). I know a way to prevent (1) on the slave (and it is implemented in the patches). I do not know if there is a way to prevent (3), it would require understanding better how innodb autoincrement and partitioning works. I do not see any way to prevent (2) from occuring on the slave, in the general case. It also seems to me that there are likely to be more corner cases lurking, and even if I somehow managed to fix the three above problems, there would be some other cases where an extra lock conflict can occur on the slave and cause the slave to hang. So I decided that a general solution is needed to such deadlocks. So the currently proposed patch does two things: A. It adds code to InnoDB, so that whenever T1 starts to wait inside InnoDB for a lock held by T2, this wait is reported to the upper server layer. The replication code will then check if T1 and T2 are part of parallel replication such that T1 must commit before T2. If so, we have a deadlock, and the code will terminate T2 with a deadlock error, allowing T1 to proceed. B. It implements code to handle deadlocks during parallel replication. When a transaction gets a deadlock error (normal, or provoked by the new deadlock detection code from (B)), it is rolled back, and re-executed. The result should be that even if there are other non-discovered cases where locks can conflict in different ways on the master from on the parallel slave, such conflicts will not cause replication to fail. Instead they will be detected, and handled automatically by resolving the deadlock. I think this will be important to ensure the reliability of parallel replication for users. I went through quite a few different ideas and attempts before ending with the present solution. So I wanted to give this background about how the patch relates to the basic problem that needs to be solved. I found a _lot_ of tricky issues during testing, and so I am somewhat unhappy about the complexity of this, even though the final patch may look simple enough on the surface. Maybe you can have some ideas for what could be done simpler, or at least question anything that looks incorrect or unnecessary. Thanks, - Kristian.

2 3

[Maria-developers] Debian packaging - option to enable feedback plugin
by Colin Charles 04 Jul '14

04 Jul '14

Hi! This is probably a great idea: https://answers.launchpad.net/maria/+question/248639 Let's ask dpkg to optionally enable feedback plugin? This could be good for us Just to be clear: I'm referring to this for the packages at mariadb.org I don't know if Debian upstream would like that -- Otto, any thoughts? cheers, -colin -- Colin Charles, Chief Evangelist, SkySQL - The MariaDB Company blog: http://bytebot.net/blog/| t: +6-012-204-3201 | Skype: colincharles

5 5

Re: [Maria-developers] Review request for MDEV-6020, MDEV-5914, MDEV-5941, and MDEV-5262
by Kristian Nielsen 04 Jul '14

04 Jul '14

Jan Lindström <jan.lindstrom(a)skysql.com> writes: > 1) In innodb deadlock detection it is not certain that there is actually a > deadlock, deadlock detection is done when a lock request is issued. At that > point yes, we can see from lock manager that there is a deadlock, however > if waiting transaction is rolled back e.g. by user request concurrently, > the deadlock detected is not valid anymore. Hm, interesting. It was my impression that the deadlock detection algorithm only considers transactions that are already waiting for a row lock. That is, if we detect a simple deadlock between two transactions T1 and T2, then it is because T2 is waiting for a row lock held by T1 while executing some user query, and T1 is executing another user query that requests another row lock held by T2. So since both T1 and T2 are already executing user queries, and will not complete until the deadlock is resolved, how could the user roll either of them back concurrently? > 2) It is fine to roll back all transactions participating to deadlock, this > is correct. Note that there are some preferences which transaction is > rolled back on normal InnoDB server and this could be different when we > have Galera > 3) How this new needs to wait work with Galera where we have so called > brute force transactions that may not wait anything ? Note that this is only for resolving deadlocks between replicated transactions from a single master. If T1 and T2 are both replicated from the same master, we might decide to kill T2 to let T1 commit first. If I understand correctly, "brute force" transactions are only for transactions that are replicated from other Galera nodes, and which _must_ be committed in a specific order to preserve correctness. Thus, a brute-force transaction would not be affected by a deadlock kill from parallel replication. Galera code might cause a parallel replication transaction to be rolled back and retried, but this should be fine. Note also that in general for MySQL/MariaDB style replication, it is assumed (and it is the user's responsibility to ensure it) that replicated transactions will not conflict with anything done on the local node. This might also help to provide that Galera and parallel replication will not conflict. If Galera had been written with a proper general high-level API that could work with different storage-engines, then we could probably have used the "brute force transaction" mechanism to solve this problem in parallel replication also. But when I checked it, it seemed that it was hardcoded inside InnoDB with lots of Galera-specific code and data structures. If work were to start to fix this lack of proper API in Galera, then perhaps it would be interesting to work together to design a general API for enforcing a specific commit order... - Kristian.

1 0

[Maria-developers] [GSoC] optimize mysql-test runs - Preparing next steps
by Pablo Estrada 04 Jul '14

04 Jul '14

Hello Elena, I understand that the recall that we are achieving so far should still be improved; nonetheless, since we are already in the second half of the summer, I was wondering if we should spend some time designing the interfaces for the implementation of the project; such that I can start building them (even though the research phase can't be marked off as completed yet). Regards Pablo

2 1

Re: [Maria-developers] MDEV-5756 CMake option to build without thread pool
by Sergei Golubchik 03 Jul '14

03 Jul '14

Hi, Holyfoot! On Jul 01, holyfoot(a)askmonty.org wrote: > revno: 4270 > revision-id: holyfoot(a)askmonty.org-20140701104158-4gj5ztnjqlr1alwa > parent: holyfoot(a)askmonty.org-20140630193024-56wpcvwwy24kuzt4 > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: mdev-5756 > timestamp: Tue 2014-07-01 15:41:58 +0500 > message: > MDEV-5756 CMake option to build without thread pool. > Check if the threadpool is available on the system and set HAVE_POOL_OF_THREADS respectively. > === modified file 'sql/CMakeLists.txt' > --- a/sql/CMakeLists.txt 2014-05-30 11:24:25 +0000 > +++ b/sql/CMakeLists.txt 2014-07-01 10:41:58 +0000 > @@ -30,7 +30,15 @@ ${CMAKE_CURRENT_BINARY_DIR}/lex_hash.h > > SET_SOURCE_FILES_PROPERTIES(${GEN_SOURCES} PROPERTIES GENERATED 1) > > -ADD_DEFINITIONS(-DMYSQL_SERVER -DHAVE_EVENT_SCHEDULER -DHAVE_POOL_OF_THREADS) > +ADD_DEFINITIONS(-DMYSQL_SERVER -DHAVE_EVENT_SCHEDULER) > + > +IF (CMAKE_SYSTEM_NAME MATCHES "Linux" OR > + CMAKE_SYSTEM_NAME MATCHES "Windows" OR > + CMAKE_SYSTEM_NAME MATCHES "SunOS" OR > + HAVE_KQUEUE) > + ADD_DEFINITIONS(-DHAVE_POOL_OF_THREADS) > +ENDIF() > + > IF(SSL_DEFINES) > ADD_DEFINITIONS(${SSL_DEFINES}) > ENDIF() Did you test it on any of the platforms that that didn't support thread pool? If you would've done it, you'd noticed that your patch doesn't work. Because scheduler.h contains: #undef HAVE_POOL_OF_THREADS #if !defined(EMBEDDED_LIBRARY) && !defined(_AIX) #define HAVE_POOL_OF_THREADS 1 So, HAVE_POOL_OF_THREADS is only disabled in embedded and on AIX. The AIX part was added recently by Bar, as a quick fix to make MariaDB to compile on AIX. Your patch should remove the AIX test from scheduler.h. Actually, you should remove all #define and #undef HAVE_POOL_OF_THREADS from all files, only CMakeLists.txt should do it. Otherwise it's ok. I don't like that it explicitly checks for OS (Linux, Windows, SunOS) instead of checking for specific functions, libraries, or headers. But it's the same check that threadpool_unix.cc does, so it's ok. Regards, Sergei

1 0

[Maria-developers] [GSoC] Optimize mysql-test-runs - Analyzing some data
by Pablo Estrada 02 Jul '14

02 Jul '14

Hello Elena and all, I have spent a couple of days writing up some scripts to analyze the data, and try to find good candidates for metrics to improve recall, or some vices in the way the relevance is calculated for tests, such that they are not close enough in the priority queue to be spotted and run. Here's a writedown of some of the findings: *Of the failures that we miss, how many of them come from tests that have never failed before?* I added a small function in the simulator.py code, to keep diagnostics data of the test failures that we missed. I made a few charts regarding missed failures of tests that have never failed before (and, in the current model, we can't calculate a relevancy for them, because we have no data about them). In the next chart, we observe the percentage of missed failures that come from tests that had never failed before increases with time (as expected): By the way, this info comes from a standard run with 3000 learning cycles and 2000 prediction cycles. I feel that this is an important area. If we can figure a way to predict failures of tests that have not failed before, then we can predict subsequent failures of these tests, and thus improve recall significantly, specially over the long term. On the next section I analyze the option that I considered. *Is there a relationship between a test file being modified and the test failing any soon?* I figured that a good idea to catch failures of tests that had never failed before could be using the test file changes. If a test file is changed, I figured that the test itself would change behavior and might fail soon, and thus we could gather some data about that failure. I coded a small script that computes the number of test runs between a test file being edited, and the same test failing sometime in the future. The relationship, unfortunately is not very strong. I have attached the chart with the cummulative histogram of number of test runs that occurred between a test file being edited, and the next failure of that test. Unfortunately, the results were not too exciting. It turns out that tests don't tend to fail too soon after they have been changed (I guess this also makes sense). The following figures present the data (and a zoom in of the part of the graph we care most about), showing that failures are not very subsequent to test file editions. Not even 10% of failures occur in the first 100 runs. As we can see from the charts, there is not a very significant relationship between test file edition and subsequent failure. It seems this indicator won't be too useful. I am* still looking for ideas* on how to catch tests failing for the first time. They are all welcome : ) This is all the data that I collected so far. I am also trying to answer the following question: Failures of tests with non-zero relevance. How come we are missing those that we are missing? I have collected and charted some data, but I still haven't found anything that feels conclusive or revealing. I expect to write an email with that info soon as well, but I decided to send this out first. In conclusion: - I am thinking about how to predict first failures of a test. Any ideas? - I tested the relation between edition of a test result file and subsequent test failure. It was not a good predictor. Regards Pablo

1 0

[Maria-developers] [GSoC] Week Six!
by sriram patil 30 Jun '14

30 Jun '14

Hi all, Following is the blog link about my sixth week into GSoC. http://goo.gl/1dzgHe Suggestions and/or reviews are welcome! Thanks, Sriram

3 2

Re: [Maria-developers] [Commits] Rev 3855: Bumping server version. (10.0.13-galera) in lp:~maria-captains/maria/maria-10.0-galera
by Roberto Spadim 30 Jun '14

30 Jun '14

=] nice nirbhay 2014-06-30 10:33 GMT-03:00 Nirbhay Choubey <nirbhay(a)skysql.com>: > Hi, > > On Mon, Jun 30, 2014 at 9:13 AM, Roberto Spadim <roberto(a)spadim.com.br> > wrote: >> >> this will be a production ready version? > > > 10.0.12-galera (released over weekend) is the first 10.0-galera stable > release. This 'patch version bump' is > for preparation towards the next release. > > Best, > Nirbhay > >> >> >> 2014-06-30 10:03 GMT-03:00 Nirbhay Choubey <nirbhay(a)skysql.com>: >> > At lp:~maria-captains/maria/maria-10.0-galera >> > >> > ------------------------------------------------------------ >> > revno: 3855 >> > revision-id: nirbhay(a)skysql.com-20140630130329-w8kiirtf3blli7l1 >> > parent: nirbhay(a)skysql.com-20140626161112-q1iqtoyq5owutvvb >> > committer: Nirbhay Choubey <nirbhay(a)skysql.com> >> > branch nick: maria-10.0-galera >> > timestamp: Mon 2014-06-30 09:03:29 -0400 >> > message: >> > Bumping server version. (10.0.13-galera) >> > >> > === modified file 'VERSION' >> > --- a/VERSION 2014-06-19 17:12:38 +0000 >> > +++ b/VERSION 2014-06-30 13:03:29 +0000 >> > @@ -4,5 +4,5 @@ >> > # >> > MYSQL_VERSION_MAJOR=10 >> > MYSQL_VERSION_MINOR=0 >> > -MYSQL_VERSION_PATCH=12 >> > +MYSQL_VERSION_PATCH=13 >> > MYSQL_VERSION_EXTRA= >> > >> > >> > _______________________________________________ >> > commits mailing list >> > commits(a)mariadb.org >> > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits >> >> >> >> -- >> Roberto Spadim >> SPAEmpresarial >> Eng. Automação e Controle >> _______________________________________________ >> commits mailing list >> commits(a)mariadb.org >> https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits > > > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

Re: [Maria-developers] [Commits] Rev 3855: Bumping server version. (10.0.13-galera) in lp:~maria-captains/maria/maria-10.0-galera
by Roberto Spadim 30 Jun '14

30 Jun '14

this will be a production ready version? 2014-06-30 10:03 GMT-03:00 Nirbhay Choubey <nirbhay(a)skysql.com>: > At lp:~maria-captains/maria/maria-10.0-galera > > ------------------------------------------------------------ > revno: 3855 > revision-id: nirbhay(a)skysql.com-20140630130329-w8kiirtf3blli7l1 > parent: nirbhay(a)skysql.com-20140626161112-q1iqtoyq5owutvvb > committer: Nirbhay Choubey <nirbhay(a)skysql.com> > branch nick: maria-10.0-galera > timestamp: Mon 2014-06-30 09:03:29 -0400 > message: > Bumping server version. (10.0.13-galera) > > === modified file 'VERSION' > --- a/VERSION 2014-06-19 17:12:38 +0000 > +++ b/VERSION 2014-06-30 13:03:29 +0000 > @@ -4,5 +4,5 @@ > # > MYSQL_VERSION_MAJOR=10 > MYSQL_VERSION_MINOR=0 > -MYSQL_VERSION_PATCH=12 > +MYSQL_VERSION_PATCH=13 > MYSQL_VERSION_EXTRA= > > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

[Maria-developers] Merge request: Cross-compile fixes (MySQL bug #61340)
by James Le Cuirot 30 Jun '14

30 Jun '14

Hello all, I posted a merge request to Launchpad back in April. A Launchpad glitch meant that it didn't get any attention but I was told on IRC to post about it here. The merge request: https://code.launchpad.net/~chewi/maria/mysql-fix-bug61340/+merge/217351 The commit diff: http://bazaar.launchpad.net/~chewi/maria/mysql-fix-bug61340/revision/4165 Given the nature of the fix, I can appreciate that it is difficult for other developers to test but I can tell you that I have just tried out a cross-built MariaDB under QEMU/proot and it seems to work. I have also seen posts from others stating that they made similar temporary adjustments to the CMake files with successful results. I am providing this contribution to the MariaDB Foundation under the MCA and also wish for it to be applied to MySQL. Regards, James

2 2

[Maria-developers] Connector Coding till 29th June
by Puneet Dewan 29 Jun '14

29 Jun '14

Hi Everyone, This week I fixed the null bitmap and types that were being sent to the server. So now I am able to successfully insert multiple values in the DB. Currently only Integer and String. This week 1.I will extend all the TypeParameter classes and ParameterWriter class so as to send the value in binary format. 2.Also try insert on different data types. 3. Write test cases Currently I am extending the PacketOutputStream.java class. Tomorrow I will have code review with Massimo. Regards Puneet.

1 0

[Maria-developers] TIME/DATETIME/TIMESTAMP data types with fractional precision
by Alexander Barkov 26 Jun '14

26 Jun '14

Hello, I propose to switch to MySQL-5.6 TIME/DATETIME/TIMESTAMP formats by default and deprecate MariaDB native temporal data types, together with pre-5.6 non-fractional types. The problem is that MariaDB native data types (and MySQL-5.5 non-fractional data types) do not work with row based replication in case of: TIME(N) -> TIME(M) There is no metadata telling about the column precision in the binary log file. It can only work when N=M. MySQL-5.6 uses separate type codes for the temporal data types: MYSQL_TYPE_TIME2 MYSQL_TYPE_DATETIME2 MYSQL_TYPE_TIMESTAMP2 and does put precision into metadata in the binary log. So it works for N != M. There is a JIRA task for this: MDEV-5377 Row-based replication of MariaDB temporal data types with FSP>0 into a different column type I'd propose to do it in 10.1.x, as 10.0.x is too late. Greetings.

2 1

Re: [Maria-developers] [Commits] Rev 4245: MDEV-6336: mysqldump --master-data does not work with GTID setups in http://bazaar.launchpad.net/~maria-captains/maria/10.0
by Sergey Vojtovich 25 Jun '14

25 Jun '14

Hi Kristian, MDEV-6344 looks fine. But I don't feel confident reviewing MDEV-6336. Regards, Sergey On Mon, Jun 16, 2014 at 11:38:04AM +0200, knielsen(a)knielsen-hq.org wrote: > At http://bazaar.launchpad.net/~maria-captains/maria/10.0 > > ------------------------------------------------------------ > revno: 4245 > revision-id: knielsen(a)knielsen-hq.org-20140616093803-gub60pda6t77ch3k > parent: igor(a)askmonty.org-20140610223256-nqs0kkp7i45qzyvx > committer: knielsen(a)knielsen-hq.org > branch nick: work-10.0 > timestamp: Mon 2014-06-16 11:38:03 +0200 > message: > MDEV-6336: mysqldump --master-data does not work with GTID setups > MDEV-6344: mysqldump issues FLUSH TABLES, which gets written into binlog and replicated > > Add a --gtid option (for compatibility, the original behaviour is preserved > when --gtid is not used). > > With --gtid, --master-data and --dump-slave output the GTID position (the > old-style file/offset position is still output, but commented out). Also, a > CHANGE MASTER TO master_use_gtid=slave_pos is output to ensure a provisioned > slave is configured in GTID, as requested. > > Without --gtid, the GTID position is still output, if available, but commented > out. > > Also fix MDEV-6344, to avoid FLUSH TABLES getting into the binlog. Otherwise a > mysqldump on a slave server will silently inject a GTID which does not exist > on the master, which is highly undesirable. > === modified file 'client/client_priv.h' > --- a/client/client_priv.h 2013-09-14 01:09:36 +0000 > +++ b/client/client_priv.h 2014-06-16 09:38:03 +0000 > @@ -92,6 +92,7 @@ enum options_client > OPT_REPORT_PROGRESS, > OPT_SKIP_ANNOTATE_ROWS_EVENTS, > OPT_SSL_CRL, OPT_SSL_CRLPATH, > + OPT_USE_GTID, > OPT_MAX_CLIENT_OPTION /* should be always the last */ > }; > > > === modified file 'client/mysqldump.c' > --- a/client/mysqldump.c 2014-05-09 10:35:11 +0000 > +++ b/client/mysqldump.c 2014-06-16 09:38:03 +0000 > @@ -134,6 +134,7 @@ static ulong opt_compatible_mode= 0; > #define MYSQL_OPT_SLAVE_DATA_COMMENTED_SQL 2 > static uint opt_mysql_port= 0, opt_master_data; > static uint opt_slave_data; > +static uint opt_use_gtid; > static uint my_end_arg; > static char * opt_mysql_unix_port=0; > static int first_error=0; > @@ -346,6 +347,13 @@ static struct my_option my_long_options[ > {"force", 'f', "Continue even if we get an SQL error.", > &ignore_errors, &ignore_errors, 0, GET_BOOL, NO_ARG, > 0, 0, 0, 0, 0, 0}, > + {"gtid", OPT_USE_GTID, "Used together with --master-data=1 or --dump-slave=1." > + "When enabled, the output from those options will set the GTID position " > + "instead of the binlog file and offset; the file/offset will appear only as " > + "a comment. When disabled, the GTID position will still appear in the " > + "output, but only commented.", > + &opt_use_gtid, &opt_use_gtid, 0, GET_BOOL, NO_ARG, > + 0, 0, 0, 0, 0, 0}, > {"help", '?', "Display this help message and exit.", 0, 0, 0, GET_NO_ARG, > NO_ARG, 0, 0, 0, 0, 0, 0}, > {"hex-blob", OPT_HEXBLOB, "Dump binary strings (BINARY, " > @@ -1177,7 +1185,7 @@ check_consistent_binlog_pos(char *binlog > > if (mysql_query_with_error_report(mysql, &res, > "SHOW STATUS LIKE 'binlog_snapshot_%'")) > - return 1; > + return 0; > > found= 0; > while ((row= mysql_fetch_row(res))) > @@ -1200,6 +1208,90 @@ check_consistent_binlog_pos(char *binlog > return (found == 2); > } > > + > +/* > + Get the GTID position corresponding to a given old-style binlog position. > + This uses BINLOG_GTID_POS(). The advantage is that the GTID position can > + be obtained completely non-blocking in this way (without the need for > + FLUSH TABLES WITH READ LOCK), as the old-style position can be obtained > + with START TRANSACTION WITH CONSISTENT SNAPSHOT. > + > + Returns 0 if ok, non-zero if error. > +*/ > +static int > +get_binlog_gtid_pos(char *binlog_pos_file, char *binlog_pos_offset, > + char *out_gtid_pos) > +{ > + DYNAMIC_STRING query; > + MYSQL_RES *res; > + MYSQL_ROW row; > + int err; > + char file_buf[FN_REFLEN*2+1], offset_buf[LONGLONG_LEN*2+1]; > + size_t len_pos_file= strlen(binlog_pos_file); > + size_t len_pos_offset= strlen(binlog_pos_offset); > + > + if (len_pos_file >= FN_REFLEN || len_pos_offset > LONGLONG_LEN) > + return 0; > + mysql_real_escape_string(mysql, file_buf, binlog_pos_file, len_pos_file); > + mysql_real_escape_string(mysql, offset_buf, binlog_pos_offset, len_pos_offset); > + init_dynamic_string_checked(&query, "SELECT BINLOG_GTID_POS('", 256, 1024); > + dynstr_append_checked(&query, file_buf); > + dynstr_append_checked(&query, "', '"); > + dynstr_append_checked(&query, offset_buf); > + dynstr_append_checked(&query, "')"); > + > + err= mysql_query_with_error_report(mysql, &res, query.str); > + dynstr_free(&query); > + if (err) > + return err; > + > + err= 1; > + if ((row= mysql_fetch_row(res))) > + { > + strmake(out_gtid_pos, row[0], FN_REFLEN-1); > + err= 0; > + } > + mysql_free_result(res); > + > + return err; > +} > + > + > +/* > + Get the GTID position on a master or slave. > + The parameter MASTER is non-zero to get the position on a master > + (@@gtid_binlog_pos) or zero for a slave (@@gtid_slave_pos). > + > + This uses the @@gtid_binlog_pos or @@gtid_slave_pos, so requires FLUSH TABLES > + WITH READ LOCK or similar to be consistent. > + > + Returns 0 if ok, non-zero for error. > +*/ > +static int > +get_gtid_pos(char *out_gtid_pos, int master) > +{ > + MYSQL_RES *res; > + MYSQL_ROW row; > + int found; > + > + if (mysql_query_with_error_report(mysql, &res, > + (master ? > + "SELECT @@GLOBAL.gtid_binlog_pos" : > + "SELECT @@GLOBAL.gtid_slave_pos"))) > + return 1; > + > + found= 0; > + if ((row= mysql_fetch_row(res))) > + { > + strmake(out_gtid_pos, row[0], FN_REFLEN-1); > + found++; > + } > + mysql_free_result(res); > + > + return (found != 1); > +} > + > + > static char *my_case_str(const char *str, > uint str_len, > const char *token, > @@ -4799,12 +4891,14 @@ static int dump_selected_tables(char *db > } /* dump_selected_tables */ > > > -static int do_show_master_status(MYSQL *mysql_con, int consistent_binlog_pos) > +static int do_show_master_status(MYSQL *mysql_con, int consistent_binlog_pos, > + int have_mariadb_gtid, int use_gtid) > { > MYSQL_ROW row; > MYSQL_RES *UNINIT_VAR(master); > char binlog_pos_file[FN_REFLEN]; > char binlog_pos_offset[LONGLONG_LEN+1]; > + char gtid_pos[FN_REFLEN]; > char *file, *offset; > const char *comment_prefix= > (opt_master_data == MYSQL_OPT_MASTER_DATA_COMMENTED_SQL) ? "-- " : ""; > @@ -4815,6 +4909,9 @@ static int do_show_master_status(MYSQL * > return 1; > file= binlog_pos_file; > offset= binlog_pos_offset; > + if (have_mariadb_gtid && > + get_binlog_gtid_pos(binlog_pos_file, binlog_pos_offset, gtid_pos)) > + return 1; > } > else > { > @@ -4844,6 +4941,9 @@ static int do_show_master_status(MYSQL * > return 0; > } > } > + > + if (have_mariadb_gtid && get_gtid_pos(gtid_pos, 1)) > + return 1; > } > > /* SHOW MASTER STATUS reports file and position */ > @@ -4852,7 +4952,19 @@ static int do_show_master_status(MYSQL * > "recovery from\n--\n\n"); > fprintf(md_result_file, > "%sCHANGE MASTER TO MASTER_LOG_FILE='%s', MASTER_LOG_POS=%s;\n", > - comment_prefix, file, offset); > + (use_gtid ? "-- " : comment_prefix), file, offset); > + if (have_mariadb_gtid) > + { > + print_comment(md_result_file, 0, > + "\n--\n-- GTID to start replication from\n--\n\n"); > + if (use_gtid) > + fprintf(md_result_file, > + "%sCHANGE MASTER TO MASTER_USE_GTID=slave_pos;\n", > + comment_prefix); > + fprintf(md_result_file, > + "%sSET GLOBAL gtid_slave_pos='%s';\n", > + (!use_gtid ? "-- " : comment_prefix), gtid_pos); > + } > check_io(md_result_file); > > if (!consistent_binlog_pos) > @@ -4922,12 +5034,16 @@ static int add_slave_statements(void) > return(0); > } > > -static int do_show_slave_status(MYSQL *mysql_con) > +static int do_show_slave_status(MYSQL *mysql_con, int use_gtid, > + int have_mariadb_gtid) > { > MYSQL_RES *UNINIT_VAR(slave); > MYSQL_ROW row; > const char *comment_prefix= > (opt_slave_data == MYSQL_OPT_SLAVE_DATA_COMMENTED_SQL) ? "-- " : ""; > + const char *gtid_comment_prefix= (use_gtid ? comment_prefix : "-- "); > + const char *nogtid_comment_prefix= (!use_gtid ? comment_prefix : "-- "); > + int set_gtid_done= 0; > > if (mysql_query_with_error_report(mysql_con, &slave, > multi_source ? > @@ -4945,8 +5061,30 @@ static int do_show_slave_status(MYSQL *m > > while ((row= mysql_fetch_row(slave))) > { > + if (multi_source && !set_gtid_done) > + { > + char gtid_pos[FN_REFLEN]; > + if (have_mariadb_gtid && get_gtid_pos(gtid_pos, 0)) > + return 1; > + if (opt_comments) > + fprintf(md_result_file, "\n--\n-- Gtid position to start replication " > + "from\n--\n\n"); > + fprintf(md_result_file, "%sSET GLOBAL gtid_slave_pos='%s';\n", > + gtid_comment_prefix, gtid_pos); > + set_gtid_done= 1; > + } > if (row[9 + multi_source] && row[21 + multi_source]) > { > + if (use_gtid) > + { > + if (multi_source) > + fprintf(md_result_file, "%sCHANGE MASTER '%.80s' TO " > + "MASTER_USE_GTID=slave_pos;\n", gtid_comment_prefix, row[0]); > + else > + fprintf(md_result_file, "%sCHANGE MASTER TO " > + "MASTER_USE_GTID=slave_pos;\n", gtid_comment_prefix); > + } > + > /* SHOW MASTER STATUS reports file and position */ > if (opt_comments) > fprintf(md_result_file, > @@ -4955,9 +5093,9 @@ static int do_show_slave_status(MYSQL *m > > if (multi_source) > fprintf(md_result_file, "%sCHANGE MASTER '%.80s' TO ", > - comment_prefix, row[0]); > + nogtid_comment_prefix, row[0]); > else > - fprintf(md_result_file, "%sCHANGE MASTER TO ", comment_prefix); > + fprintf(md_result_file, "%sCHANGE MASTER TO ", nogtid_comment_prefix); > > if (opt_include_master_host_port) > { > @@ -5030,12 +5168,13 @@ static int do_flush_tables_read_lock(MYS > FLUSH TABLES is to lower the probability of a stage where both mysqldump > and most client connections are stalled. Of course, if a second long > update starts between the two FLUSHes, we have that bad stall. > + > + We use the LOCAL option, as we do not want the FLUSH TABLES replicated to > + other servers. > */ > return > - ( mysql_query_with_error_report(mysql_con, 0, > - ((opt_master_data != 0) ? > - "FLUSH /*!40101 LOCAL */ TABLES" : > - "FLUSH TABLES")) || > + ( mysql_query_with_error_report(mysql_con, 0, > + "FLUSH /*!40101 LOCAL */ TABLES") || > mysql_query_with_error_report(mysql_con, 0, > "FLUSH TABLES WITH READ LOCK") ); > } > @@ -5656,6 +5795,7 @@ int main(int argc, char **argv) > char bin_log_name[FN_REFLEN]; > int exit_code; > int consistent_binlog_pos= 0; > + int have_mariadb_gtid= 0; > MY_INIT(argv[0]); > > sf_leaking_memory=1; /* don't report memory leaks on early exits */ > @@ -5696,7 +5836,10 @@ int main(int argc, char **argv) > > /* Check if the server support multi source */ > if (mysql_get_server_version(mysql) >= 100000) > + { > multi_source= 2; > + have_mariadb_gtid= 1; > + } > > if (opt_slave_data && do_stop_slave_sql(mysql)) > goto err; > @@ -5743,9 +5886,11 @@ int main(int argc, char **argv) > /* Add 'STOP SLAVE to beginning of dump */ > if (opt_slave_apply && add_stop_slave()) > goto err; > - if (opt_master_data && do_show_master_status(mysql, consistent_binlog_pos)) > + if (opt_master_data && do_show_master_status(mysql, consistent_binlog_pos, > + have_mariadb_gtid, opt_use_gtid)) > goto err; > - if (opt_slave_data && do_show_slave_status(mysql)) > + if (opt_slave_data && do_show_slave_status(mysql, opt_use_gtid, > + have_mariadb_gtid)) > goto err; > if (opt_single_transaction && do_unlock_tables(mysql)) /* unlock but no commit! */ > goto err; > > === modified file 'mysql-test/r/rpl_mysqldump_slave.result' > --- a/mysql-test/r/rpl_mysqldump_slave.result 2014-02-26 14:28:07 +0000 > +++ b/mysql-test/r/rpl_mysqldump_slave.result 2014-06-16 09:38:03 +0000 > @@ -4,18 +4,59 @@ include/master-slave.inc > # New --dump-slave, --apply-slave-statements functionality > # > use test; > +-- SET GLOBAL gtid_slave_pos=''; > CHANGE MASTER '' TO MASTER_LOG_FILE='master-bin.000001', MASTER_LOG_POS=BINLOG_START; > STOP ALL SLAVES; > +-- SET GLOBAL gtid_slave_pos=''; > CHANGE MASTER '' TO MASTER_LOG_FILE='master-bin.000001', MASTER_LOG_POS=BINLOG_START; > START ALL SLAVES; > STOP ALL SLAVES; > +-- SET GLOBAL gtid_slave_pos=''; > CHANGE MASTER '' TO MASTER_HOST='127.0.0.1', MASTER_PORT=MASTER_MYPORT, MASTER_LOG_FILE='master-bin.000001', MASTER_LOG_POS=BINLOG_START; > START ALL SLAVES; > start slave; > Warnings: > Note 1254 Slave is already running > +-- SET GLOBAL gtid_slave_pos=''; > CHANGE MASTER '' TO MASTER_LOG_FILE='master-bin.000001', MASTER_LOG_POS=BINLOG_START; > start slave; > Warnings: > Note 1254 Slave is already running > +*** Test mysqldump --dump-slave GTID functionality. > +SET gtid_seq_no = 1000; > +CREATE TABLE t1 (a INT PRIMARY KEY); > +DROP TABLE t1; > +CREATE TABLE t2 (a INT PRIMARY KEY); > +DROP TABLE t2; > + > +1. --dump-slave=1 > + > +SET GLOBAL gtid_slave_pos='0-1-1001'; > +CHANGE MASTER '' TO MASTER_USE_GTID=slave_pos; > +-- CHANGE MASTER '' TO MASTER_LOG_FILE='master-bin.000001', MASTER_LOG_POS=BINLOG_START; > + > +2. --dump-slave=2 > + > +-- SET GLOBAL gtid_slave_pos='0-1-1001'; > +-- CHANGE MASTER '' TO MASTER_USE_GTID=slave_pos; > +-- CHANGE MASTER '' TO MASTER_LOG_FILE='master-bin.000001', MASTER_LOG_POS=BINLOG_START; > +*** Test mysqldump --master-data GTID functionality. > + > +1. --master-data=1 > + > +-- CHANGE MASTER TO MASTER_LOG_FILE='slave-bin.000001', MASTER_LOG_POS=BINLOG_START; > +CHANGE MASTER TO MASTER_USE_GTID=slave_pos; > +SET GLOBAL gtid_slave_pos='0-2-1003'; > + > +2. --master-data=2 > + > +-- CHANGE MASTER TO MASTER_LOG_FILE='slave-bin.000001', MASTER_LOG_POS=BINLOG_START; > +-- CHANGE MASTER TO MASTER_USE_GTID=slave_pos; > +-- SET GLOBAL gtid_slave_pos='0-2-1003'; > + > +3. --master-data --single-transaction > + > +-- CHANGE MASTER TO MASTER_LOG_FILE='slave-bin.000001', MASTER_LOG_POS=BINLOG_START; > +CHANGE MASTER TO MASTER_USE_GTID=slave_pos; > +SET GLOBAL gtid_slave_pos='0-2-1003'; > include/rpl_end.inc > > === modified file 'mysql-test/t/rpl_mysqldump_slave.test' > --- a/mysql-test/t/rpl_mysqldump_slave.test 2014-02-26 14:28:07 +0000 > +++ b/mysql-test/t/rpl_mysqldump_slave.test 2014-06-16 09:38:03 +0000 > @@ -36,4 +36,53 @@ start slave; > --exec $MYSQL_DUMP_SLAVE --compact --dump-slave no_such_db > start slave; > > + > +--echo *** Test mysqldump --dump-slave GTID functionality. > + > +--connection master > +SET gtid_seq_no = 1000; > +CREATE TABLE t1 (a INT PRIMARY KEY); > +DROP TABLE t1; > +--sync_slave_with_master > + > +--connection slave > +# Inject a local transaction on the slave to check that this is not considered > +# for --dump-slave. > +CREATE TABLE t2 (a INT PRIMARY KEY); > +DROP TABLE t2; > + > +--echo > +--echo 1. --dump-slave=1 > +--echo > +--replace_regex /MASTER_LOG_POS=[0-9]+/MASTER_LOG_POS=BINLOG_START/ > +--exec $MYSQL_DUMP_SLAVE --compact --dump-slave=1 --gtid test > + > +--echo > +--echo 2. --dump-slave=2 > +--echo > +--replace_regex /MASTER_LOG_POS=[0-9]+/MASTER_LOG_POS=BINLOG_START/ > +--exec $MYSQL_DUMP_SLAVE --compact --dump-slave=2 --gtid test > + > + > +--echo *** Test mysqldump --master-data GTID functionality. > +--echo > +--echo 1. --master-data=1 > +--echo > +--replace_regex /MASTER_LOG_POS=[0-9]+/MASTER_LOG_POS=BINLOG_START/ > +--exec $MYSQL_DUMP_SLAVE --compact --master-data=1 --gtid test > + > +--echo > +--echo 2. --master-data=2 > +--echo > +--replace_regex /MASTER_LOG_POS=[0-9]+/MASTER_LOG_POS=BINLOG_START/ > +--exec $MYSQL_DUMP_SLAVE --compact --master-data=2 --gtid test > + > +--echo > +--echo 3. --master-data --single-transaction > +--echo > +--replace_regex /MASTER_LOG_POS=[0-9]+/MASTER_LOG_POS=BINLOG_START/ > +--exec $MYSQL_DUMP_SLAVE --compact --master-data --single-transaction --gtid test > + > + > + > --source include/rpl_end.inc > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

2 1

Re: [Maria-developers] [Commits] Rev 4259: MDEV-6120: When slave stops with error, error message should indicate the failing GTID in http://bazaar.launchpad.net/~maria-captains/maria/10.0
by Sergey Vojtovich 25 Jun '14

25 Jun '14

Hi Kristian, looks good. One minor question below. On Thu, Jun 19, 2014 at 02:50:44PM +0200, knielsen(a)knielsen-hq.org wrote: > At http://bazaar.launchpad.net/~maria-captains/maria/10.0 > > ------------------------------------------------------------ > revno: 4259 > revision-id: knielsen(a)knielsen-hq.org-20140619125043-acw2gx7rw90y3vqp > parent: svoj(a)mariadb.org-20140618141504-kqq6ws9gm93r2b2d > committer: Kristian Nielsen <knielsen(a)knielsen-hq.org> > branch nick: work-10.0 > timestamp: Thu 2014-06-19 14:50:43 +0200 > message: > MDEV-6120: When slave stops with error, error message should indicate the failing GTID > > If replication breaks in GTID mode, it is not trivial to determine the GTID of > the failing event group. This is a problem, as such GTID is needed eg. to > explicitly set @@gtid_slave_pos to skip to after that event group, or to > compare errors on different servers, etc. > > Fix by ensuring that relevant slave errors logged to the error log include the > GTID of the event group containing the problem event. ...skip... > === added file 'mysql-test/suite/rpl/t/rpl_gtid_errorlog.test' > --- a/mysql-test/suite/rpl/t/rpl_gtid_errorlog.test 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/suite/rpl/t/rpl_gtid_errorlog.test 2014-06-19 12:50:43 +0000 > @@ -0,0 +1,94 @@ > +--source include/have_debug.inc > +--source include/master-slave.inc > + > +--echo *** Test MDEV-6120, output of current GTID when a replication error is logged to the errorlog *** > +--connection master > +CREATE TABLE t1(a INT PRIMARY KEY); > +--sync_slave_with_master > + > +--connection slave > +--source include/stop_slave.inc > +CHANGE MASTER TO master_use_gtid=slave_pos; > + > +--connection master > +INSERT INTO t1 VALUES (1); > +SET gtid_seq_no=100; > +INSERT INTO t1 VALUES (2); > +INSERT INTO t1 VALUES (3); > +INSERT INTO t1 VALUES (4); > +--save_master_pos > + > +--connection slave > +SET sql_log_bin=0; > +INSERT INTO t1 VALUES (2); > +SET sql_log_bin=1; > + > +START SLAVE; > +--let $slave_sql_errno=1062 > +--source include/wait_for_slave_sql_error.inc > + > +--source include/stop_slave.inc > +# Skip the problem event from the master. > +SET GLOBAL gtid_slave_pos= "0-1-100"; > +--source include/start_slave.inc > +--sync_with_master > + > +SELECT * FROM t1 ORDER BY a; > + > +--connection master > + > +SET @dbug_save= @@debug_dbug; > +SET debug_dbug= '+d,incident_database_resync_on_replace'; > +REPLACE INTO t1 VALUES (5); > +SET debug_dbug= @dbug_save; > +--save_master_pos > + > +--connection slave > +--let $slave_sql_errno=1590 > +--source include/wait_for_slave_sql_error.inc > +# ToDo no need to switch off GTID once MDEV-4937 is fixed > +--source include/stop_slave.inc > +CHANGE MASTER TO master_use_gtid=no; > +SET sql_slave_skip_counter=1; > +--source include/start_slave.inc > +--sync_with_master > +--source include/stop_slave.inc > +CHANGE MASTER TO master_use_gtid=slave_pos; > +--source include/start_slave.inc > + > +SELECT * FROM t1 ORDER BY a; > + > + > +# Check error log for correct messages. > +let $log_error_= `SELECT @@GLOBAL.log_error`; > +if(!$log_error_) > +{ > + # MySQL Server on windows is started with --console and thus > + # does not know the location of its .err log, use default location > + let $log_error_ = $MYSQLTEST_VARDIR/log/mysqld.2.err; > +} > +let LOG_ERROR=$log_error_; > + > +perl; > + use strict; > + use warnings; > + my $log_error= $ENV{'LOG_ERROR'} or die "LOG_ERROR not set"; > + open(FILE, "<", "$log_error") or die("Unable to open $log_error: $!\n"); > + my $found_dup= 0; > + my $found_incident= 0; > + while (<FILE>) { > + $found_dup= 1 > + if /Slave SQL: Error 'Duplicate entry .* on query\. .*Query: '.*', Gtid 0-1-100, Internal MariaDB error code:/ > + or /Slave SQL: Could not execute Write_rows.*table test.t1; Duplicate entry.*, Gtid 0-1-100, Internal MariaDB error/ > + ; > + $found_incident= 1 > + if /Slave SQL: The incident LOST_EVENTS occured on the master\. Message: <none>, Internal MariaDB error code: 1590/; > + } > + print "Duplicate key found: $found_dup; incident found: $found_incident\n"; > + close(FILE); > +EOF > + > +--connection master > +DROP TABLE t1; > + > +--source include/rpl_end.inc Why didn't you like include/search_pattern_in_file.inc? It should perfectly able to match your patterns. ...skip... Regards, Sergey

2 1

[Maria-developers] Review of MDEV-5730: enhance security using special compilation options
by Kristian Nielsen 24 Jun '14

24 Jun '14

> commit feab48657528b9bb40fb035d51bee28d93710c1e > Author: Sergei Golubchik <serg(a)mariadb.org> > Date: Mon Jun 16 21:39:09 2014 +0200 > > MDEV-5730 enhance security using special compilation options > > -Wl,-z,relro,-z,now > -pie > -fstack-protector --param=ssp-buffer-size=4 > -D_FORTIFY_SOURCE=2 The patch is ok to push with a clarified comment as requested below. I have checked the compiler/linker options added and commented some on them below. > diff --git a/CMakeLists.txt b/CMakeLists.txt > index a24f000..5b0d348 100644 > --- a/CMakeLists.txt > +++ b/CMakeLists.txt > @@ -201,6 +201,20 @@ IF (WITH_ASAN) > ENDIF() > ENDIF() > > +OPTION(SECURITY_HARDENED "Use security-enhancing compiler features (stack protector, relro, etc)" ON) So these are on by default. This means that this patch will cause some performance impact. So having them on by default is a compromise between performance and security. You should probably include in the commit comment a discussion of why the security viewpoint was prefered over the performance viewpoint in this case (or a comment in the cmake file if you prefer). The proper way to handle a patch like this would be to measure the performance impact, eg. for single-threaded performance with some various simple sysbench loads. However, it is quite hard to do such measurements, as you need a setup that has low enough noise in results to detect sub-percentage differences of performance. I have had some success by locking processes to a single core and using `perf` measurements of cycles spent (rather than wall-clock time), but even so the results can be tricky to interpret. So it may be too much work for a small patch like this. (And even if the performance improvement can be measured, how can we measure the security impact to weight them against one another?) My personal opinion is that I do not buy into this "security" mindset much. I do not think these options improve security significantly in practise for most users. So while the performance impact is probably also not very significant, why accept it at all (by default)? But the opposite viewpoint exists and is valid as well. So I think this patch is ok if you add a discussion of why it was chosen to do it this way. (Even if the reason is "Most distros do it" - at least then it is documented what the reason is). Also, the option should mention the possible performance impact. Suggestion: "Use compiler options (stack protector, relro, etc) that may slightly enhance security of the resulting binary, at the possible cost of some small performance impact." > +IF(SECURITY_HARDENED) > + # security-enhancing flags > + MY_CHECK_AND_SET_COMPILER_FLAG("-pie -fPIC") -fPIC can have a significant impact on performance on some platforms. For example on 32-bit x86, it requires extra instructions at the head of every function, and it makes one less register available for code generation on a CPU already starved for registers. On 64-bit x86, the impact is much smaller, as the architecture was enhanced to better support -fPIC. However, I seem to recall that the server is anyway built with -fPIC, for libmysqld or something like that? In which case this has no extra impact. > + MY_CHECK_AND_SET_COMPILER_FLAG("-Wl,-z,relro,-z,now") This option seems to cause all relocations to happen at load time, rather than lazily on-demand. The impact should be higher memory usage and some extra CPU time for these relocations, many of which may never be needed. This option will have the biggest effect on the client programs probably, as server startup is already fairly heavy. Probably even for client programs the added cost will be not very significant, as even the client programs are usually called to do significant processing (eg. connect to a database and so on). I guess these options would mostly hurt small utility programs for shell scripts, like `mv` or so. > + MY_CHECK_AND_SET_COMPILER_FLAG("-fstack-protector --param=ssp-buffer-size=4") This also adds extra code and memory loads to each function, so it will have some impact to performance. It is probably the most significant impact of all the added options, though without detailed analysis this is only guesswork, of course. > + # sometimes _FORTIFY_SOURCE is predefined > + INCLUDE(CheckSymbolExists) > + CHECK_SYMBOL_EXISTS(_FORTIFY_SOURCE "" HAVE_FORTIFY_SOURCE) > + IF(NOT HAVE_FORTIFY_SOURCE) > + ADD_DEFINITIONS(-D_FORTIFY_SOURCE=2) > + ENDIF() > +ENDIF() FORTIFY_SOURCE also adds some runtime overhead, but it is not very much, it is an extra argument to memcpy() and friends about destination buffer size. Probably the way or code is written, there will be few places where this option makes any difference, so the performance impact does not seem likely to be very significant. Hope this helps, - Kristian.

3 2

[Maria-developers] [GSoC] Optimize mysql-test-runs - Setback
by Pablo Estrada 24 Jun '14

24 Jun '14

Hello Sergei, Elena and all, Today while working on the script, I found and fixed an issue: There is some faulty code code in my script that is in charge of collecting the statistics about whether a test failure was caught or not (here <https://github.com/pabloem/Kokiri/blob/master/basic_simulator.py#L393>). I looked into fixing it, and then I could see another *problem*: The *recall numbers* that I had collected previously were *too high*. The actual recall numbers, once we consider the test failures that are *not caught*, are disappointingly lower. I won't show you results yet, since I want to make sure that the code has been fixed, and I have accurate tests first. This is all for now. The strategy that I was using is a lot less effective than it seemed initially. I will send out a more detailed report with results, my opinion on the weak points of the strategy, and ideas, including a roadmap to try to improve results. Regards. All feedback is welcome. Pablo

3 18

[Maria-developers] Doubt in bin logging for CREATE VIEW
by sriram patil 24 Jun '14

24 Jun '14

Hi All, I was checking the behavior of bin logging in various methods. I found out CREATE VIEW differs a little bit than others in bin logging. The query is written into bin log if query is executed successfully. If there is any error the query is not logged. This behavior is observed across all the commands. In case of CREATE VIEW (method: mysql_create_view file: sql/sql_view.cc), while the bin logging is skipped for most of the errors by jumping to "err" label, it is not skipped in following code snippet (line number: 616) res= mysql_register_view(thd, view, mode); /* View TABLE_SHARE must be removed from the table definition cache in order to make ALTER VIEW work properly. Otherwise, we would not be able to detect meta-data changes after ALTER VIEW. */ if (!res) tdc_remove_table(thd, TDC_RT_REMOVE_ALL, view->db, view->table_name, false); if (mysql_bin_log.is_open()) { // this is where the log is written } If an error occurs in mysql_register_view, e.g. ER_OUT_OF_RESOURCES, ER_TABLE_EXISTS_ERROR, etc. the query is still logged. Is there a specific reason why it is logged even though an error occurs? Or is it a bug? Thanks & Regards, Sriram

4 9

Re: [Maria-developers] [Commits] 7d386bb: MDEV-4549 [PATCH] Clean up code working with ACL tables
by Sergey Vojtovich 23 Jun '14

23 Jun '14

Hi Sergei, ok to push. A few minor questions inline. On Mon, Jun 16, 2014 at 09:34:21AM +0200, serg(a)mariadb.org wrote: > revision-id: 7d386bbce216745ba28df728f45a0b796fcced98 > parent(s): f61f36b386e8d0c6448297bb84c92e8d9b82be6a > committer: Sergei Golubchik > branch nick: maria > timestamp: 2014-06-15 17:12:57 +0200 > message: > > MDEV-4549 [PATCH] Clean up code working with ACL tables > > enum values to index different ACL tables, instead of hard-coded numbers > (even different in diffent functions). Also factor out TABLE_LIST > initialization into a separate function. > > --- > mysql-test/r/not_embedded_server.result | 4 +- > mysql-test/r/sp-destruct.result | 3 - > .../suite/rpl/r/rpl_tmp_table_and_DDL.result | 22 +- > mysql-test/t/not_embedded_server.test | 2 +- > mysql-test/t/sp-destruct.test | 1 - > sql/sql_acl.cc | 533 +++++++++------------ > 6 files changed, 235 insertions(+), 330 deletions(-) > > diff --git a/mysql-test/r/not_embedded_server.result b/mysql-test/r/not_embedded_server.result > index 2295276..6dda855 100644 > --- a/mysql-test/r/not_embedded_server.result > +++ b/mysql-test/r/not_embedded_server.result > @@ -1,4 +1,4 @@ > -call mtr.add_suppression("Can't open and lock privilege tables: Table 'host' was not locked with LOCK TABLES"); > +call mtr.add_suppression("Can't open and lock privilege tables: Table 'roles_mapping' was not locked with LOCK TABLES"); > SHOW VARIABLES like 'slave_skip_errors'; > Variable_name Value > slave_skip_errors OFF Why did it change (here and in other tests)? I didn't notice any table order change in the code. ...skip... > diff --git a/mysql-test/r/sp-destruct.result b/mysql-test/r/sp-destruct.result > index 172e40c..fe68229 100644 > --- a/mysql-test/r/sp-destruct.result > +++ b/mysql-test/r/sp-destruct.result > @@ -128,11 +128,8 @@ CREATE FUNCTION f1() RETURNS INT RETURN 1; > RENAME TABLE mysql.procs_priv TO procs_priv_backup; > FLUSH TABLE mysql.procs_priv; > DROP FUNCTION f1; > -ERROR 42S02: Table 'mysql.procs_priv' doesn't exist > SHOW WARNINGS; > Level Code Message > -Error 1146 Table 'mysql.procs_priv' doesn't exist > -Warning 1405 Failed to revoke all privileges to dropped routine > # Restore the procs_priv table > RENAME TABLE procs_priv_backup TO mysql.procs_priv; > FLUSH TABLE mysql.procs_priv; Hmm... why? ...skip... > diff --git a/sql/sql_acl.cc b/sql/sql_acl.cc > index 5e8e0e7..442d512 100644 > --- a/sql/sql_acl.cc > +++ b/sql/sql_acl.cc > @@ -380,7 +380,7 @@ class ACL_PROXY_USER :public ACL_ACCESS > MYSQL_PROXIES_PRIV_PROXIED_USER, > MYSQL_PROXIES_PRIV_WITH_GRANT, > MYSQL_PROXIES_PRIV_GRANTOR, > - MYSQL_PROXIES_PRIV_TIMESTAMP } old_acl_proxy_users; > + MYSQL_PROXIES_PRIV_TIMESTAMP } proxy_table_fields; > public: > ACL_PROXY_USER () {}; > > @@ -760,6 +760,99 @@ static int traverse_role_graph_down(ACL_USER_BASE *, void *, > int (*) (ACL_USER_BASE *, ACL_ROLE *, void *)); > > /* > + Enumeration of ACL/GRANT tables in the mysql database > +*/ > +enum enum_acl_tables > +{ > + USER_TABLE, > + DB_TABLE, > + TABLES_PRIV_TABLE, > + COLUMNS_PRIV_TABLE, > +#define FIRST_OPTIONAL_TABLE HOST_TABLE > + HOST_TABLE, > + PROCS_PRIV_TABLE, > + PROXIES_PRIV_TABLE, > + ROLES_MAPPING_TABLE, > + TABLES_MAX // <== always the last > +}; > +// bits for init_priv_tables > +static const int Table_user= 1 << USER_TABLE; > +static const int Table_db= 1 << DB_TABLE; > +static const int Table_tables_priv= 1 << TABLES_PRIV_TABLE; > +static const int Table_columns_priv= 1 << COLUMNS_PRIV_TABLE; > +static const int Table_host= 1 << HOST_TABLE; > +static const int Table_procs_priv= 1 << PROCS_PRIV_TABLE; > +static const int Table_proxies_priv= 1 << PROXIES_PRIV_TABLE; > +static const int Table_roles_mapping= 1 << ROLES_MAPPING_TABLE; > + > +const LEX_STRING acl_table_names[]= // matches enum_acl_tables > +{ > + { C_STRING_WITH_LEN("user") }, > + { C_STRING_WITH_LEN("db") }, > + { C_STRING_WITH_LEN("tables_priv") }, > + { C_STRING_WITH_LEN("columns_priv") }, > + { C_STRING_WITH_LEN("host") }, > + { C_STRING_WITH_LEN("procs_priv") }, > + { C_STRING_WITH_LEN("proxies_priv") }, > + { C_STRING_WITH_LEN("roles_mapping") } > +}; > + > +/** > + Initialize a TABLE_LIST array to a set of acl/grant tables. > + All tables will be opened with the same lock type, either read or write. > + > + @retval !0 a pointer to the first initialized element in the TABLE_LIST > + @retval 0 replication filters matched. Abort operation, but return OK (!) > +*/ > +TABLE_LIST *init_priv_tables(THD *thd, TABLE_LIST *tables, > + enum thr_lock_type lock_type, int tables_to_open) Nice clean-up, I assume there was a reason not rewrite open_grant_tables() instead. > +{ > + int prev= -1; > + bzero(tables, sizeof(TABLE_LIST) * TABLES_MAX); > + for (int cur=0, mask=1; mask <= tables_to_open; cur++, mask <<= 1) > + { > + if ((tables_to_open & mask) == 0) > + continue; > + tables[cur].init_one_table(C_STRING_WITH_LEN("mysql"), > + acl_table_names[cur].str, > + acl_table_names[cur].length, > + acl_table_names[cur].str, lock_type); > + tables[cur].open_type= OT_BASE_ONLY; > + if (lock_type >= TL_WRITE_ALLOW_WRITE) > + tables[cur].updating= 1; > + if (cur >= FIRST_OPTIONAL_TABLE) > + tables[cur].open_strategy= TABLE_LIST::OPEN_IF_EXISTS; > + if (prev != -1) > + tables[cur].next_local= tables[cur].next_global= & tables[prev]; > + prev= cur; > + } > + > +#ifdef HAVE_REPLICATION > + if (lock_type >= TL_WRITE_ALLOW_WRITE && thd->slave_thread && !thd->spcont) > + { > + /* > + GRANT and REVOKE are applied the slave in/exclusion rules as they are > + some kind of updates to the mysql.% tables. > + */ > + Rpl_filter *rpl_filter= thd->system_thread_info.rpl_sql_info->rpl_filter; > + if (rpl_filter->is_on() && !rpl_filter->tables_ok(0, tables)) > + return NULL; > + } > +#endif Was it a bug that mysql_grant_role() didn't have this code? open_grant_tables() does updating= 0 after tables_ok(). Is that needed? ...skip... > @@ -5783,8 +5796,9 @@ bool mysql_routine_grant(THD *thd, TABLE_LIST *table_list, bool is_proc, > } > } > > - if (replace_routine_table(thd, grant_name, tables[1].table, *Str, > - db_name, table_name, is_proc, rights, > + if (no_such_table(tables + PROCS_PRIV_TABLE) || > + replace_routine_table(thd, grant_name, tables[PROCS_PRIV_TABLE].table, > + *Str, db_name, table_name, is_proc, rights, > revoke_grant) != 0) > { > result= TRUE; Why no_such_table() here? ...skip... > @@ -6250,7 +6232,7 @@ bool mysql_grant(THD *thd, const char *db, List <LEX_USER> &list, > ulong db_rights= rights & DB_ACLS; > if (db_rights == rights) > { > - if (replace_db_table(tables[1].table, db, *Str, db_rights, > + if (replace_db_table(tables[DB_TABLE].table, db, *Str, db_rights, > revoke_grant)) > result= -1; > } Tabs here (and in a few hunks below). > @@ -6262,9 +6244,11 @@ bool mysql_grant(THD *thd, const char *db, List <LEX_USER> &list, > } > else if (is_proxy) > { > - if (replace_proxies_priv_table (thd, tables[1].table, Str, proxied_user, > - rights & GRANT_ACL ? TRUE : FALSE, > - revoke_grant)) > + if (no_such_table(tables + PROXIES_PRIV_TABLE) || > + replace_proxies_priv_table (thd, tables[PROXIES_PRIV_TABLE].table, > + Str, proxied_user, > + rights & GRANT_ACL ? TRUE : FALSE, > + revoke_grant)) > result= -1; > } > if (Str->is_role()) Why no_such_table() here? ...skip... Regards, Sergey

2 2

[Maria-developers] [GSoC] Week Five!
by sriram patil 23 Jun '14

23 Jun '14

Hi All, Following is the link to my blog post of fifth week into GSoC. http://goo.gl/x7dtcn Suggestions and/or review are welcome! Thanks, Sriram

1 0

[Maria-developers] Java Connector Coding till 21st June
by Puneet Dewan 21 Jun '14

21 Jun '14

Hi Everyone. Test case class(ServersidePreparedStatementTest.java) was written for checking the value of statement id. This week code review was done by Massimo and Georg. There pointers were really valuable as data was not being send in a proper format to the server for execute(com_stmt_execute) method : 1.NullBitMap was set incorrectly. 2.Creation of a new class extending PacketOuptutStream e.g integer of 4 bytes can be sent.Currently if an integer is sent it is sent as a byte. 3.Also I was expecting when we do a *prepare* for a statement like insert into XYZ(id) values(?) where id is of type Integer. Then server shall be returning integer as a type. This was a false assumption.It does not return the type. So now on the basis of value set for *id as example*,decide on the type to be sent from execute. So this week will fix the above things. Looking forward for suggestions. Regards Puneet.

1 0

Re: [Maria-developers] [Commits] 22f532c: MDEV-4260 Don't create frm files for temporary tables
by Sergey Vojtovich 20 Jun '14

20 Jun '14

Hi Sergei, ok to push. One note inline. On Fri, Jun 20, 2014 at 11:29:36AM +0200, serg(a)mariadb.org wrote: > revision-id: 22f532cfb2f40132e2f7b079a3aafb109af92c26 > parent(s): fb8818c1aff9ce67b74f965c019cad93addd811e > committer: Sergei Golubchik > branch nick: maria > timestamp: 2014-06-20 11:29:12 +0200 > message: > > MDEV-4260 Don't create frm files for temporary tables > > * Don't write frm for tmp tables > * pass frm image down to open_table_uncached, when possible > * don't use truncate-by-recreate for temp tables - cannot recreate > without frm, and delete_all_rows is faster anyway > * remove HTON_FLUSH_AFTER_RENAME (BDB-ism, unused for years) > > --- > mysql-test/r/temp_table_frm.result | 20 +++++++++++ > mysql-test/t/temp_table_frm.test | 13 ++++++++ > sql/handler.h | 2 +- > sql/sql_base.cc | 14 +++++++- > sql/sql_base.h | 3 +- > sql/sql_table.cc | 27 +++------------ > sql/sql_truncate.cc | 68 +------------------------------------- > sql/unireg.cc | 3 +- > 8 files changed, 56 insertions(+), 94 deletions(-) > ...skip... > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > index 0558066..7d69ccb 100644 > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -5561,7 +5563,17 @@ TABLE *open_table_uncached(THD *thd, handlerton *hton, > strend(saved_cache_key)+1, tmp_path); > share->db_plugin= ha_lock_engine(thd, hton); > > - if (open_table_def(thd, share, GTS_TABLE | GTS_USE_DISCOVERY)) > + /* > + Use the frm image, if possible, open the file otherwise. > + > + The image might be unavailable in ALTER TABLE, when the discovering > + engine took over the ownership (see TABLE::read_frm_image). > + */ > + int res= frm->str > + ? res= share->init_from_binary_frm_image(thd, false, frm->str, frm->length) Excessive "res="? Regards, Sergey

2 1

[Maria-developers] [GSoC] Week Four!
by sriram patil 19 Jun '14

19 Jun '14

Hi All, Sorry for a little delay in posting the blog link this week. Following is the blog link about my fourth week into GSoC. http://goo.gl/W54cvE Suggestions and/or reviews are welcome! Thanks, Sriram

4 4

Re: [Maria-developers] [Commits] Rev 4258: MDEV-6039 - WebScaleSQL patches in lp:maria/10.0
by Sergei Golubchik 18 Jun '14

18 Jun '14

Hi, Sergey! On Jun 18, Sergey Vojtovich wrote: > At lp:maria/10.0 > > ------------------------------------------------------------ > revno: 4258 > revision-id: svoj(a)mariadb.org-20140618141504-kqq6ws9gm93r2b2d > parent: svoj(a)mariadb.org-20140618081243-jxs6yo9tx9cahdv6 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0 > timestamp: Wed 2014-06-18 18:15:04 +0400 > message: > MDEV-6039 - WebScaleSQL patches > > Stop spawning dummy threads on client library initialization > > Let's revert the fix for Bug#24507. To quote Monty from 2006: > > "After 1/2 a year, when all glibc versions are updated, we can delete > this code." > > Note: The upstream glibc bug was fixed in 2006. Ok to push Regards, Sergei

1 0