- developers - lists.mariadb.org

Re: [Maria-developers] [Commits] 7320cbf: MDEV-7177 - Server hangs on shutdown after InnoDB error (main.plugin_loaderr
by Sergei Golubchik 04 Feb '15

04 Feb '15

Hi, Sergey! On Feb 02, svoj(a)mariadb.org wrote: > revision-id: 7320cbf85ddfaf7ef76c8f03718819377fd14318 > parent(s): 11deb416fe12ebfd65a1f089c7eb6087c192e2dd > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2015-02-02 15:41:49 +0400 > message: > > MDEV-7177 - Server hangs on shutdown after InnoDB error (main.plugin_loaderr > fails in buildbot) > > There was a race condition in timer functionality of query timeouts. > > Race was as following: > main thread: init_thr_timers() > timer handler thread: my_thread_init() > main thread: end_thr_timer()/timer_thread_state= ABORTING > timer handler thread: timer_thread_state= RUNNING, continue normal op > main thread: waits indefinitely for timer handler thread to go down > > The original idea of the fix is to set RUNNING state in main thread, before > starting timer handler thread. But it didn't survive further cleanups: > - removed "timer_thread_state" and used "thr_timer_inited" for this purpose > - removed unused "timer_thread_running" > - removed code responisble for "timer handler thread" shutdown synchronization, > use pthread_join() instead. That looks ok to me. The only concern is - in end_thr_timer(), the old code had a timeout waiting for a timer thread to die. pthread_join() doesn't. I don't know if that's a problem, perhaps not. Regards, Sergei

2 1

Re: [Maria-developers] [Commits] 11deb41: MDEV-7499 - System variables have broken default values on big endian
by Sergei Golubchik 04 Feb '15

04 Feb '15

Hi, Sergey! On Jan 30, svoj(a)mariadb.org wrote: > revision-id: 11deb416fe12ebfd65a1f089c7eb6087c192e2dd > parent(s): edf34f38ac4fad7996bf19cd9ac669d2a6825400 > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2015-01-30 16:13:28 +0400 > message: > > MDEV-7499 - System variables have broken default values on big endian > > INFORMATION_SCHEMA.SYSTEM_VARIABLES.DEFAULT_VALUE had broken values on > big endian. > > Default value is internally stored as longlong, while I_S references it's > pointer (longlong *) according to variable type (e.g. int, my_bool, etc). This > works well on little endian, but on big endian we always get 0 for such > variables. Ok to push, thanks! One suggestion below > diff --git a/sql/sql_plugin.cc b/sql/sql_plugin.cc > index 46849e2..1928df5 100644 > --- a/sql/sql_plugin.cc > +++ b/sql/sql_plugin.cc > @@ -3254,7 +3254,29 @@ static int pluginvar_sysvar_flags(const st_mysql_sys_var *p) > uchar* sys_var_pluginvar::real_value_ptr(THD *thd, enum_var_type type) > { > if (type == OPT_DEFAULT) > - return (uchar*)&option.def_value; > + { > + switch (plugin_var->flags & PLUGIN_VAR_TYPEMASK) { > + case PLUGIN_VAR_BOOL: > + thd->sys_var_tmp.my_bool_value= option.def_value; > + return (uchar*) &thd->sys_var_tmp.my_bool_value; > + case PLUGIN_VAR_INT: > + thd->sys_var_tmp.int_value= option.def_value; > + return (uchar*) &thd->sys_var_tmp.int_value; > + case PLUGIN_VAR_LONG: > + case PLUGIN_VAR_ENUM: > + thd->sys_var_tmp.long_value= option.def_value; > + return (uchar*) &thd->sys_var_tmp.long_value; > + case PLUGIN_VAR_LONGLONG: > + case PLUGIN_VAR_SET: > + case PLUGIN_VAR_DOUBLE: > + return (uchar*) &option.def_value; PLUGIN_VAR_LONGLONG and PLUGIN_VAR_SET - ok, but for PLUGIN_VAR_DOUBLE I'd either copy it to thd->sys_var_tmp too or add an assert, like this case PLUGIN_VAR_DOUBLE: compile_time_assert(sizeof(double) == sizeof(option.def_value)); /* fall through */ case PLUGIN_VAR_LONGLONG: case PLUGIN_VAR_SET: return (uchar*) &option.def_value; More for documentation purposes than for protection agains a wrong sizeof(double). It explains why you don't copy the value. > + case PLUGIN_VAR_STR: > + thd->sys_var_tmp.ptr_value= (void*) option.def_value; > + return (uchar*) &thd->sys_var_tmp.ptr_value; > + default: > + DBUG_ASSERT(0); > + } > + } > Regards, Sergei

2 1

Re: [Maria-developers] [Commits] 101d7eb: MDEV-4987: Sort by domain_id when list of GTIDs are output
by Kristian Nielsen 03 Feb '15

03 Feb '15

nirbhay(a)mariadb.com writes: > revision-id: 101d7eb963816514362da8a98fc7db7135181910 > committer: Nirbhay Choubey > timestamp: 2015-01-31 21:48:14 -0500 > message: > > MDEV-4987: Sort by domain_id when list of GTIDs are output > > Added logic to sort gtid list based on domain_id before > populating them in string. Added a test case. Thanks! The patch looks ok to push after fixing / considering below in-line comments: > diff --git a/mysql-test/suite/rpl/t/rpl_gtid_sort.test b/mysql-test/suite/rpl/t/rpl_gtid_sort.test > new file mode 100644 > index 0000000..d3b8c6d > --- /dev/null > +++ b/mysql-test/suite/rpl/t/rpl_gtid_sort.test > +--connection server_2 > +SHOW VARIABLES LIKE '%GTID%'; > +#SET GLOBAL gtid_slave_pos=""; This commented-out SET looks like some left-over, probably you meant to remove it? > diff --git a/sql/rpl_gtid.cc b/sql/rpl_gtid.cc > index e5620ec..a0471f3 100644 > --- a/sql/rpl_gtid.cc > +++ b/sql/rpl_gtid.cc > @@ -247,11 +247,13 @@ > { > my_hash_init(&hash, &my_charset_bin, 32, offsetof(element, domain_id), > sizeof(uint32), NULL, rpl_slave_state_free_element, HASH_UNIQUE); > + my_init_dynamic_array(&gtid_cache, sizeof(rpl_gtid), 8, 8, MYF(0)); The name 'cache' confused me slightly, as I think of a cache as something that stores some state between invocations. But it is actually just a temporary buffer used for sorting. A suggestion for a better name could be 'gtid_sort_array'. > rpl_slave_state::~rpl_slave_state() > { > + delete_dynamic(&gtid_cache); > } The other parts of rpl_slave_state are freed in rpl_slave_state::deinit(). I think this is because there is a global variable of this class, and destructors in global variables can cause various issues. So better put this delete_dynamic() into deinit() as well (for consistency, if nothing else). > @@ -705,7 +707,13 @@ class Gtid_db_intact : public Table_check_intact > return sub_id; > } > > +/* A callback used in sorting of gtid list based on domain_id. */ > +static int rpl_gtid_cmp_cb(const void *id1, const void *id2) > +{ > + return (((rpl_gtid *)id1)->domain_id - ((rpl_gtid *)id2)->domain_id); > +} I think this is wrong, you can get signed/unsigned integer overflow (eg. it will consider 0xffffffff as being less than 0x00000000). We might be able to do the subtraction in longlong with a cast (though that would still be susceptible to overflow when converting to the int return value, right?), but it's probably clearer to just do: uint32 d1= ((rpl_gtid *)id1)->domain_id; uint32 d2= ((rpl_gtid *)id2)->domain_id; if (d1 < d2) return -1; else if (d1 > f2) return 1; else return 0; You should probably also modify the test case to have a domain id > 0x80000000, just to check. Otherwise looks good, thanks! - Kristian.

2 2

[Maria-developers] custom storage backend rnd_next() implementation and caching questions
by Andras Szabo 03 Feb '15

03 Feb '15

Hi Everyone, I am developing a custom mariadb storage backend which is intended to give filesystem related information via mariadb tables. For example one could tell: CREATE TEMPORARY TABLE IF NOT EXISTS `/var/log` ENGINE=fsview; which would create a table (using the assisted discovery method) with the following schema: MariaDB [test]> describe `/var`; +-----------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-----------+--------------+------+-----+---------+-------+ | file_name | varchar(255) | YES | | NULL | | | contents | longblob | YES | | NULL | | +-----------+--------------+------+-----+---------+———+ However i have some implementation issues: 1. I am having trouble implementing rnd_next() in the backend: int ha_fsview::rnd_next(uchar *buf) { DBUG_ENTER(__PRETTY_FUNCTION__); if (!dirp) { DBUG_RETURN(HA_ERR_INTERNAL_ERROR); } if (struct dirent* entry = readdir(dirp)) { uchar *pos = buf; *pos = 0; ++pos; // file name (1bytes size + str) *pos = strlen(entry->d_name); ++pos; size_t bytes_to_copy = strlen(entry->d_name); memcpy(pos, entry->d_name, bytes_to_copy); pos+=bytes_to_copy; DBUG_PRINT("info", ("fsview filename: %s", entry->d_name)); // content (4 bytes size + content ptr) static const std::string fake_content = "hello world"; int s = fake_content.size(); memcpy(pos, &s, sizeof(s)); pos+=sizeof(s); const char* c = fake_content.c_str(); memcpy(pos, &c, sizeof(c)); pos+=sizeof(c); DBUG_RETURN(0); } else { DBUG_RETURN(HA_ERR_END_OF_FILE); } } My problem is that filenames (first field) is ok, but i cannot really see the expected content “hello world” in the second field. I am assuming that i am doing something wrong, but i cannot really find the reason. Could somebody help me, please? 2. If i am issuing: SELECT * FROM `/var`; twice, then the server is not initiating a table scan again on my backend (most likely results from the first run are stored in some cache). So if filesystem contents are changed between the two statements, then it won’t be reflected in the resultset. Of course if i do a: SELECT SQL_NO_CACHE * FROM `/var`; then everything is fine, however i would not like to put the burden on the user to specify the option to not cache the results. My question is whether it’s possible to specify in the storage backend that the results from the tables should not be cached. Thank for the help in advance -- Andras Szabo Sent with Airmail

3 7

Re: [Maria-developers] [Commits] Rev 4577: MDEV-5719: Wrong result with GROUP BY and LEFT OUTER JOIN in lp:~maria-captains/maria/10.0
by Sergey Petrunia 29 Jan '15

29 Jan '15

Hi Sergei, == TL;DR == I have debugged that LEFT JOIN problem. The fix is not pretty, but correct. I will now push it into the 10.0 tree. == Details == So, the commit changes - return (table->null_row ? TRUE : - null_ptr ? MY_TEST(null_ptr[row_offset] & null_bit) : 0); + return real_maybe_null() ? + MY_TEST(null_ptr[row_offset] & null_bit) : table->null_row; The patch changes the behavior for NULLable fields. Suppose * real_maybe_null()= TRUE * table->null_row==TRUE // the table has a NULL-complemented row * (null_ptr[row_offset] & null_bit) == FALSE // however the value // is not null. the old code will return TRUE the new code will return FALSE The new code may seem wrong at first glance (it's a NULL-complemented row, why we return FALSE from is_null()?) However, if we look where table->null_row is set to true, we will find that it is done here: inline void mark_as_null_row(TABLE *table) { table->null_row=1; table->status|=STATUS_NULL_ROW; bfill(table->null_flags,table->s->null_bytes,255); } The function also sets all NULL-bits, so normally one can assume that (table->null_row=TRUE) => ((null_ptr[..] & null_bit) == TRUE) However, there is a case when the above does not hold. It happens when we run GROUP BY query, and GROUP BY is satisfied by scanning an index which produces rows in group order. In that case, the following can occur: consider the query from the testcase: SELECT ... FROM table_one a LEFT JOIN table_two b ON a.oid=b.oid GROUP BY a.oid * we read from a row (1,3) * we make index lookup in table b and find row (3). * row combination a(1,3)-b(3) passes the WHERE clause, GROUP BY execution code records it as current row. * there are no other matches in table b, so we get back to reading table a. * we read row a(1,4) * we look for matches in table b. there are no matches, so we produce a NULL-complemented row b(NULLs) * row combination a(1,4)-b(NULLs) is passed to the GROUP BY runtime. The runtime sees that we are now in a different group, and figures that it should pass the row combination representing the previous group into query output. * It attempts to pass the row combination a(1,3)-b(3) (*) into query output. However, the current row for table b is already the NULL-complemented row. the row combination (*) is stored in the "result_field" objects (and not in "regular field" objects which refer to table->record[0]). It seems wrong that we look at result_field (which points to a row that's different from table->record[0]), and at the same time we are looking at table->null_row, which refers to the record in table->record[0]. However, "result_field" storage has no place for table->null_row bit. The fix basically means: "for NULL-able fields, field's NULL-bit should override table->null_row". Considering the above, it is correct (albeit ugly). Now, what about GROUP BY and NOT NULL fields? I discovered the following: if a table has table->maybe_null, then Field objects that refer to such table and are pointed by Item_field::result_field, are always NULLable. (code-wise, check out setup_copy_fields(), these calls: item->result_field=field->new_field(thd->mem_root,field->table, 1); item->result_field->move_field(copy->to_ptr,copy->to_null_ptr,1); result_field object of NOT NULL column is null-able. On Fri, Jan 23, 2015 at 08:25:59AM -0800, Nirbhay Choubey wrote: > At lp:~maria-captains/maria/10.0 > > ------------------------------------------------------------ > revno: 4577 > revision-id: nirbhay(a)mariadb.com-20150123162551-h88uxn780ivj5k2n > parent: sergii(a)pisem.net-20150119153032-fqrmxjr13z0p1ly0 > committer: Nirbhay Choubey <nirbhay(a)mariadb.com> > branch nick: b5719-10.0 > timestamp: Fri 2015-01-23 11:25:51 -0500 > message: > MDEV-5719: Wrong result with GROUP BY and LEFT OUTER JOIN > > Merged revision 5224 from mysql-5.6 and added a test case. > .. > revno: 5224 > committer: Sergey Glukhov <sergey.glukhov(a)oracle.com> > branch nick: mysql-5.6 > timestamp: Wed 2013-06-19 14:24:08 +0400 > message: > Bug#16620047 INCORRECT QUERY RESULT (AFTER SERVER UPGRADE) > .. > === modified file 'mysql-test/r/group_by_innodb.result' > --- a/mysql-test/r/group_by_innodb.result 2014-01-26 20:49:11 +0000 > +++ b/mysql-test/r/group_by_innodb.result 2015-01-23 16:25:51 +0000 > @@ -57,3 +57,26 @@ > NULL 11.1,22.2 > DROP TABLE t1; > End of 5.5 tests > +# > +# MDEV-5719: Wrong result with GROUP BY and LEFT OUTER JOIN > +# > +CREATE TABLE table_one (oidGroup INT, oid INT PRIMARY KEY)ENGINE=INNODB; > +INSERT INTO table_one VALUES (1,1),(1,2),(1,3),(1,4); > +CREATE TABLE table_two (oid INT PRIMARY KEY)ENGINE=INNODB; > +INSERT INTO table_two VALUES (3); > +SELECT a.oidGroup, a.oid, b.oid FROM table_one a LEFT JOIN table_two b ON > +a.oid=b.oid WHERE a.oidGroup=1; > +oidGroup oid oid > +1 1 NULL > +1 2 NULL > +1 3 3 > +1 4 NULL > +SELECT a.oidGroup, a.oid, b.oid FROM table_one a LEFT JOIN table_two b ON > +a.oid=b.oid WHERE a.oidGroup=1 GROUP BY a.oid; > +oidGroup oid oid > +1 1 NULL > +1 2 NULL > +1 3 3 > +1 4 NULL > +DROP TABLE table_one, table_two; > +# End of tests > > === modified file 'mysql-test/t/group_by_innodb.test' > --- a/mysql-test/t/group_by_innodb.test 2014-01-26 20:49:11 +0000 > +++ b/mysql-test/t/group_by_innodb.test 2015-01-23 16:25:51 +0000 > @@ -67,3 +67,22 @@ > > --echo End of 5.5 tests > > +--echo # > +--echo # MDEV-5719: Wrong result with GROUP BY and LEFT OUTER JOIN > +--echo # > +CREATE TABLE table_one (oidGroup INT, oid INT PRIMARY KEY)ENGINE=INNODB; > +INSERT INTO table_one VALUES (1,1),(1,2),(1,3),(1,4); > + > +CREATE TABLE table_two (oid INT PRIMARY KEY)ENGINE=INNODB; > +INSERT INTO table_two VALUES (3); > + > +# Returns a value > +SELECT a.oidGroup, a.oid, b.oid FROM table_one a LEFT JOIN table_two b ON > +a.oid=b.oid WHERE a.oidGroup=1; > + > +SELECT a.oidGroup, a.oid, b.oid FROM table_one a LEFT JOIN table_two b ON > +a.oid=b.oid WHERE a.oidGroup=1 GROUP BY a.oid; > + > +DROP TABLE table_one, table_two; > + > +--echo # End of tests > > === modified file 'sql/field.h' > --- a/sql/field.h 2014-10-20 12:42:00 +0000 > +++ b/sql/field.h 2015-01-23 16:25:51 +0000 > @@ -656,21 +656,28 @@ > inline bool is_null(my_ptrdiff_t row_offset= 0) const > { > /* > + If the field is NULLable, it returns NULLity based > + on null_ptr[row_offset] value. Otherwise it returns > + NULL flag depending on TABLE::null_row value. > + > The table may have been marked as containing only NULL values > for all fields if it is a NULL-complemented row of an OUTER JOIN > or if the query is an implicitly grouped query (has aggregate > functions but no GROUP BY clause) with no qualifying rows. If > - this is the case (in which TABLE::null_row is true), the field > - is considered to be NULL. > + this is the case (in which TABLE::null_row is true) and the > + field is not nullable, the field is considered to be NULL. > + > + Do not change the order of testing. Fields may be associated > + with a TABLE object without being part of the current row. > + For NULL value check to work for these fields, they must > + have a valid null_ptr, and this pointer must be checked before > + TABLE::null_row. > + > Note that if a table->null_row is set then also all null_bits are > set for the row. > - > - Otherwise, if the field is NULLable, it has a valid null_ptr > - pointer, and its NULLity is recorded in the "null_bit" bit of > - null_ptr[row_offset]. > */ > - return (table->null_row ? TRUE : > - null_ptr ? MY_TEST(null_ptr[row_offset] & null_bit) : 0); > + return real_maybe_null() ? > + MY_TEST(null_ptr[row_offset] & null_bit) : table->null_row; > } > inline bool is_real_null(my_ptrdiff_t row_offset= 0) const > { return null_ptr ? (null_ptr[row_offset] & null_bit ? 1 : 0) : 0; } > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

2 2

[Maria-developers] Dynamic columns support in libmaria client library?
by Peter Laursen 28 Jan '15

28 Jan '15

This page https://mariadb.com/kb/en/mariadb/documentation/nosql/dynamic-columns-api/ does not mention anything about it. it just says *The API is a part of libmysql C client library. In order to use it, one needs to include this header file* *#include <mysql/ma_dyncol.h>* *and link against libmysql.* Now can ma_dyncol.h be linked against libmaria as well? There are 2 questions actually 1) technical: will it work with libmaria? 2) legal: can a client program using libmaria under LGPL license include ma_dyncol.h? -- Peter -- Webyog

3 5

Re: [Maria-developers] [Commits] 51fa027: Improve performance for calculating memory allocation
by Sergei Golubchik 25 Jan '15

25 Jan '15

Hi, Monty! On Jan 23, Michael Widenius wrote: > revision-id: 51fa0272febd782674b3d44828718e805c44c1a8 > parent(s): ab440b0fb7302d707ba0ba41382bf911404db1cb > committer: Michael Widenius > branch nick: maria-10.1 > timestamp: 2015-01-23 14:32:53 +0200 > message: > > Improve performance for calculating memory allocation See my comments below: > diff --git a/include/mysql/plugin.h b/include/mysql/plugin.h > index 16b0c6e..9ae0fe8 100644 > --- a/include/mysql/plugin.h > +++ b/include/mysql/plugin.h > @@ -190,6 +196,7 @@ struct st_mysql_show_var { > > #define SHOW_VAR_FUNC_BUFF_SIZE (256 * sizeof(void*)) > typedef int (*mysql_show_var_func)(MYSQL_THD, struct st_mysql_show_var*, char *); > +typedef int (*mysql_show_var_extended_func)(MYSQL_THD, struct st_mysql_show_var*, enum enum_var_type, char *); I'd put scope the last making mysql_show_var_extended_func backward compatible with mysql_show_var_func. this makes some code simpler, see below. > > /* > diff --git a/sql/sql_show.cc b/sql/sql_show.cc > index 044b60d..8eed7b9 100644 > --- a/sql/sql_show.cc > +++ b/sql/sql_show.cc > @@ -3115,9 +3115,16 @@ static bool show_status_array(THD *thd, const char *wild, > if var->type is SHOW_FUNC or SHOW_SIMPLE_FUNC, call the function. > Repeat as necessary, if new var is again one of the above > */ > - for (var=variables; var->type == SHOW_FUNC || > - var->type == SHOW_SIMPLE_FUNC; var= &tmp) > - ((mysql_show_var_func)(var->value))(thd, &tmp, buff); > + for (var=variables; > + var->type == SHOW_FUNC || var->type == SHOW_SIMPLE_FUNC || > + var->type == SHOW_EXTENDED_FUNC; > + var= &tmp) > + { > + if (var->type == SHOW_EXTENDED_FUNC) > + ((mysql_show_var_extended_func)(var->value))(thd, &tmp, scope, buff); > + else > + ((mysql_show_var_func)(var->value))(thd, &tmp, buff); You won't need this if() when scope is the last argument. In fact I'm still thinking whether it would be a good idea to add scope to mysql_show_var_func and SHOW_FUNC, without introducing SHOW_EXTENDED_FUNC at all. This wouldn't break any existing plugin, as it's perfectly binary compatible extension. Also, see below. > + } > > SHOW_TYPE show_type=var->type; > if (show_type == SHOW_ARRAY) > diff --git a/sql/sql_class.cc b/sql/sql_class.cc > index 0d49407..d200af6 100644 > --- a/sql/sql_class.cc > +++ b/sql/sql_class.cc > @@ -1747,6 +1749,16 @@ void add_to_status(STATUS_VAR *to_var, STATUS_VAR *from_var) > to_var->binlog_bytes_written+= from_var->binlog_bytes_written; > to_var->cpu_time+= from_var->cpu_time; > to_var->busy_time+= from_var->busy_time; > + to_var->local_memory_used+= from_var->local_memory_used; why? it's not additive, right? and global_status_var.local_memory_used is never shown anyway. > + > + /* > + Update global_memory_used. We have to do this with atomic_add as the > + global value can change outside of LOCK_status. > + */ > + // workaround for gcc 4.2.4-1ubuntu4 -fPIE (from DEB_BUILD_HARDENING=1) > + int64 volatile * volatile ptr= &to_var->global_memory_used; I'd rather remove this workaround. 4.2.4 is ancient, and quite probably this bug doesn't happen with MY_MEMORY_ORDER_RELAXED at all > + my_atomic_add64_explicit(ptr, from_var->global_memory_used, > + MY_MEMORY_ORDER_RELAXED); > } > > /* > @@ -1784,6 +1796,11 @@ void add_diff_to_status(STATUS_VAR *to_var, STATUS_VAR *from_var, > dec_var->binlog_bytes_written; > to_var->cpu_time+= from_var->cpu_time - dec_var->cpu_time; > to_var->busy_time+= from_var->busy_time - dec_var->busy_time; > + > + /* > + We don't need to accumulate memory_used as these are not after > + this function is called. Sorry, I cannot parse that :( > + */ > } > > #define SECONDS_TO_WAIT_FOR_KILL 2 > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index ea53e47..c4e1dc3 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -3972,19 +3972,24 @@ static void my_malloc_size_cb_func(long long size, my_bool is_thread_specific) > if (thd) > { > DBUG_PRINT("info", ("memory_used: %lld size: %lld", > - (longlong) thd->status_var.memory_used, size)); > + (longlong) thd->status_var.local_memory_used, > + size)); > - thd->status_var.memory_used+= size; > + thd->status_var.local_memory_used+= size; > - DBUG_ASSERT((longlong) thd->status_var.memory_used >= 0); > + DBUG_ASSERT((longlong) thd->status_var.local_memory_used >= 0); > } > } > } > + else if (likely(thd)) > + thd->status_var.global_memory_used+= size; Hmm, ok. So, you think that current_thd is cheaper than updating a global variable? > + else > + { > // workaround for gcc 4.2.4-1ubuntu4 -fPIE (from DEB_BUILD_HARDENING=1) > - int64 volatile * volatile ptr=&global_status_var.memory_used; > + int64 volatile * volatile ptr=&global_status_var.global_memory_used; I would've removed this workaround too > my_atomic_add64_explicit(ptr, size, MY_MEMORY_ORDER_RELAXED); > + } > } > } > > - > static int init_common_variables() > { > umask(((~my_umask) & 0666)); > @@ -8028,6 +8033,21 @@ static int show_default_keycache(THD *thd, SHOW_VAR *var, char *buff) > return 0; > } > > + > +static int show_memory_used(THD *thd, SHOW_VAR *var, enum enum_var_type scope, > + char *buff) > +{ > + var->type= SHOW_LONGLONG; > + var->value= buff; > + if (scope == OPT_GLOBAL) > + *(longlong*) buff= (global_status_var.local_memory_used + > + global_status_var.global_memory_used); > + else > + *(longlong*) buff= thd->status_var.local_memory_used; > + return 0; I'm surprised you haven't added SHOW_EXTENDED_SIMPLE_FUNC :) That would've make a very confusing name though. May be we should indeed add scope as the last argument to mysql_show_var_func and reuse both SHOW_SIMPLE_FUNC and SHOW_FUNC? I'm starting to think it's a good thing to do. Makes the API simpler and the number of choices to understand - smaller. If you do that - please don't forget to increment MARIA_PLUGIN_INTERFACE_VERSION. > +} > + > + > #ifndef DBUG_OFF > static int debug_status_func(THD *thd, SHOW_VAR *var, char *buff) > { Regards, Sergei

1 0

[Maria-developers] Additions to GIS in 10.1.
by Alexey Botchkov 23 Jan '15

23 Jan '15

Hi all. What's new about GIS in 10.1. Related tasks in JIRA are: MDEV-4045 Missing OGC Spatial functions. MDEV-60 Support for Spatial Reference systems for the GIS data. MDEV-12 OpenGIS: create required tables: GeometryColumns, related views. Speaking shortly, the MariaDB GIS part is now OpenGIS compliant, and passes all the OpenGIS conformance tests To get that the next things were done: Missing functions were added: IsRing(ln) The return type is Integer, with a return value of 1 for TRUE, 0 for FALSE, and –1 for UNKNOWN corresponding to a function invocation on NULL arguments; return TRUE if c is a ring, i.e., if c is closed and simple. A simple Curve does not pass through the same Point more than once. PointOnSurface(s) The return type is POINT, returns a Point guaranteed to lie on the Surface. arguments: POLYGON or MYLTIPOLYGON. NumInteriorRing(p) The return type is Integer, returns the number of interiorRings arguments: POLYGON. Relate(g1 Geometry, g2 Geometry, patternMatrix String) The return type is Integer, with a return value of 1 for TRUE, 0 for FALSE, and –1 for UNKNOWN corresponding to a function invocation on NULL arguments; returns TRUE if the spatial relationship specified by the pattern Matrix holds. ConvexHull(g1 Geometry) The return is Geometry, returna a geometric object that is the convex hull of g1 arguments: GEOMETRY. The information about the GEOMETRY column's reference system (SRID) is now stored in the field's metadata. Syntax of the CREATE and ALTER was modified to specify the SRID for the geometry columns. With this information some views required by OpenGIS, were added: GEOMETRY_COLUMNS, SPATIAL_REF_SYS: These are links to the INFORMATION_SCHEMA.GEOMETRY_COLUMNS and INFORMATION_SCHEMA.SPATIAL_REF_SYS respectively. INFORMATION_SCHEMA.GEOMETRY_COLUMNS: describes the available feature tables and their Geometry properties. Fields: F_TABLE_CATALOG CHARACTER VARYING(256) NOT NULL, F_TABLE_SCHEMA CHARACTER VARYING(256) NOT NULL, F_TABLE_NAME CHARACTER VARYING(256) NOT NULL, the above 3 fields are the fully qualified name of the feature table containing the geometry column; F_GEOMETRY_COLUMN CHARACTER VARYING(256) NOT NULL, the name of the column in the feature table that is the Geometry Column. G_TABLE_CATALOG CHARACTER VARYING(256) NOT NULL, G_TABLE_SCHEMA CHARACTER VARYING(256) NOT NULL, G_TABLE_NAME CHARACTER VARYING(256) NOT NULL, the above 3 fields are the name of the geometry table and its schema and catalog. The geometry table implements the geometry column; STORAGE_TYPE INTEGER, always 1 in MariaDB (means binary geometry implementation) GEOMETRY_TYPE INTEGER, the type of geometry values stored in this column. 0 = GEOMETRY 1 = POINT 3 = LINESTRING 5 = POLYGON 7 = MULTIPOINT 9 = MULTILINESTRING 11 = MULTIPOLYGON COORD_DIMENSION INTEGER, the number of dimensions in the spatial reference system. Always 2 in MariaDB. MAX_PPR INTEGER, Always 0 in MariaDB. SRID INTEGER the ID of the Spatial Reference System used for the coordinate geometry in this table. It is a foreign key reference to the SPATIAL_REF_SYS table. INFORMATION_SCHEMA.SPATIAL_REF_SYS: stores information on each spatial reference system used in the database. Fields: SRID INTEGER NOT NULL PRIMARY KEY, an integer value that uniquely identifies each Spatial Reference System within a database. AUTH_NAME CHARACTER VARYING(256), the name of the standard or standards body that is being cited for this reference system. In most cases the value is "EPSG". AUTH_SRID INTEGER TEXT the Well-known Text Representation of the Spatial Reference System. Required stored procedures added: AddGeometryColumn(FEATURE_TABLE_CATALOG, FEATURE_TABLE_SCHEMA, FEATURE_TABLE_NAME, GEOMETRY_COLUMN_NAME, SRID) adds new column of spatial type to the specified table DropGeometryColumn(FEATURE_TABLE_CATALOG, FEATURE_TABLE_SCHEMA, FEATURE_TABLE_NAME, GEOMETRY_COLUMN_NAME) removes the spatial column from the table. That's it for now. Best regards. HF

1 0

[Maria-developers] Renaming Virtual Columns to Computed Columns
by Ian Gilfillan 19 Jan '15

19 Jan '15

There was a suggestion on the docs mailing list that we rename Virtual Columns to Computed Columns. I think it makes sense, but before we implement I want to make sure everyone is happy with the change, so am repeating it here as not everyone follows docs. Currently, we use the term Virtual Columns (see https://mariadb.com/kb/en/mariadb/documentation/sql-structure-and-commands/…) Oracle uses this term, while MS SQL and Firebird use the term Computed Columns to describe similar functionality. MariaDB's virtual column implementation has both persistent and virtual attributes. Talking of "virtual virtual" and "persistent virtual" can be confusing. Oracle's implementation, to my knowledge, only has virtual virtual, while MS SQL's Computed Columns offers both persistent ("persisted" in their case) and virtual. I think "Computed" is a slightly more descriptive term, and avoids the ambiguity of "persistent virtual" and "virtual virtual". The MariaDB error messages already use the term computed, for example "Constant expression in computed column function is not allowed". Unless anyone has a good reason to prefer the term virtual, we're going to rather use the term "computed column" in the docs. The article will still mention virtual column as an alternative etc, but computed will be the preferred term. ian

3 3

Re: [Maria-developers] [Commits] Rev 4562: MDEV-7402 'reset master' hangs, waits for signalled COND_xid_list in lp:~maria-captains/maria/10.0
by Kristian Nielsen 19 Jan '15

19 Jan '15

Sergei Golubchik <serg(a)mariadb.org> writes: > ------------------------------------------------------------ > revno: 4562 > revision-id: sergii(a)pisem.net-20150118130747-rs67yzzk8a7ebwdd > parent: sergii(a)pisem.net-20150118002113-snztyzwvzc500c2m > fixes bug: https://mariadb.atlassian.net/browse/MDEV-7402 > committer: Sergei Golubchik <sergii(a)pisem.net> > branch nick: 10.0 > timestamp: Sun 2015-01-18 14:07:47 +0100 > message: > MDEV-7402 'reset master' hangs, waits for signalled COND_xid_list > > Using a boolean flag for 'there is a RESET MASTER in progress' doesn't > work very well for multiple concurrent RESET MASTER statements. Ok to push with a testcase like the below, as we discussed. - Kristian. ----------------------------------------------------------------------- --echo *** MDEV-7402: 'reset master' hangs, waits for signalled COND_xid_list *** --source include/wait_for_binlog_checkpoint.inc connect(con3,localhost,root,,); # Make the binlog background thread wait before clearing the pending checkpoint. # The bug was that one RESET MASTER would clear the reset_master_pending # flag set by another RESET MASTER; this could cause the wakeup from the # binlog background thread not to be sent, and thus the second RESET MASTER # to wait infinitely. SET debug_sync="reset_logs_after_set_reset_master_pending SIGNAL reset_master_ready WAIT_FOR reset_master_cont"; send RESET MASTER; --connection default SET @old_dbug= @@global.DEBUG_DBUG; SET GLOBAL debug_dbug="+d,inject_binlog_background_thread_before_mark_xid_done"; SET debug_sync="now WAIT_FOR reset_master_ready"; RESET MASTER; SET debug_sync="now WAIT_FOR injected_binlog_background_thread"; SET GLOBAL debug_dbug=@old_dbug; SET debug_sync="now SIGNAL reset_master_cont"; --connection con3 REAP; --connection default SET debug_sync = 'reset';

1 0

Re: [Maria-developers] [MDEV-7362] ANALYZE TABLES crash with table-independent-statistics gathering
by Sergei Golubchik 17 Jan '15

17 Jan '15

Hi, Vicențiu! On Jan 17, Vicențiu Ciorbaru wrote: > > Here's the new diff, let me know if you have any more comments. Shall I > push this as is? Yes, just one small detail (file name and the comment), see below. Then please push, no need to ask for another review :) But... 1. Please don't paste patches into the email anymore. At least not with your mail client that wraps long lines around and sends everything as html. See https://mariadb.com/kb/en/using-git/#commit-emails 2. Why git? This is the patch for 10.0 bug, at the moment 10.0 is still in bzr. 3. This looks like a reverse diff, how did you manage to do that? :) > diff --git a/mysql-test/t/statistics_index_crash.test b/mysql-test/t/statistics_index_crash.test > deleted file mode 100644 > index dbbdca3..0000000 > --- a/mysql-test/t/statistics_index_crash.test if you create a new test file for one bug only (as opposite to adding to an existing test file), mention the bug number in the file name. Like statistics_index_crash-7362.test > +++ /dev/null > @@ -1,26 +0,0 @@ > ---source include/have_stat_tables.inc > ---source include/have_innodb.inc Start a test case for a bug with a comment that says the bug number, the synopsys, and optionally more information, if needed. The bare minimum looks like # # MDEV-7362 ANALYZE TABLES crash with table-independent-statistics gathering # MDEV-7380 engine-independent stats SEGV on ANALYZE TABLE (#2) # > - > -CREATE TABLE t1 (a longtext, FULLTEXT KEY (`a`)) ENGINE=InnoDB; > -INSERT INTO t1 VALUES (unhex('3E0D0A4141414142334E7A6143317963324541414141424977414141674541726D')); > -ANALYZE TABLE t1 PERSISTENT FOR ALL; > ---sorted_result > -SELECT * FROM mysql.index_stats WHERE index_name='a' AND table_name='t1'; > -DROP TABLE t1; > - > -CREATE TABLE t1 (a longtext, FULLTEXT KEY (`a`)) ENGINE=MyISAM; > -INSERT INTO t1 VALUES (unhex('3E0D0A4141414142334E7A6143317963324541414141424977414141674541726D')); > -ANALYZE TABLE t1 PERSISTENT FOR ALL; > ---sorted_result > -SELECT * FROM mysql.index_stats WHERE index_name='a' AND table_name='t1'; > -DROP TABLE t1; > - > -CREATE TABLE geom (g GEOMETRY NOT NULL, SPATIAL INDEX(g)) ENGINE=MyISAM; > -INSERT INTO geom VALUES > - (MultiPolygonFromText('MULTIPOLYGON(((28 26,28 0,84 0,84 42,28 26),(52 18,66 23,73 9,48 6,52 18)),((59 18,67 18,67 13,59 13,59 18)))')), > - (MPolyFromText('MULTIPOLYGON(((28 26,28 0,84 0,84 42,28 26),(52 18,66 23,73 9,48 6,52 18)),((59 18,67 18,67 13,59 13,59 18)))')), > - (MPolyFromWKB(AsWKB(MultiPolygon(Polygon(LineString(Point(0, 3), Point(3, 3), Point(3, 0), Point(0, 3))))))); > -ANALYZE TABLE geom PERSISTENT FOR ALL; > ---sorted_result > -SELECT * FROM mysql.index_stats WHERE index_name='g' AND table_name='geom'; > -DROP TABLE geom; Regards, Sergei

1 0

[Maria-developers] [MDEV-7362] ANALYZE TABLES crash with table-independent-statistics gathering
by Vicențiu Ciorbaru 17 Jan '15

17 Jan '15

Hi Sergei! I've implemented a fix for the bug described in MDEV-7362, that also fixes MDEV-7380. The test case also covers MDEV-7380. With the following fix,the I've spent a bit of time trying to figure out the architecture of things there. I am not confident my fix is complete, or if this is the way we want to go about fixing it. The fix does what we discussed over IRC, that we are not interested in computing statistics for a FULLTEXT index. What I think would be the root cause of the problem is calling ha_index_init() on an index that does not support this. By calling table->file->ha_index_init(), the table->field array gets corrupted and that ultimately leads to a crash in the function using it afterwards. Then again, this module is completely new to me, so any comments are appreciated. Here is the diff of the first changes. diff --git a/mysql-test/t/mdev-7362.test b/mysql-test/t/mdev-7362.test new file mode 100644 index 0000000..6551893 --- /dev/null +++ b/mysql-test/t/mdev-7362.test @@ -0,0 +1,14 @@ +CREATE TABLE t1 (a longtext, FULLTEXT KEY (`a`)) ENGINE=MyISAM; +insert into t1 values (unhex('3E0D0A4141414142334E7A6143317963324541414141424977414141674541726D')); +analyze table t1 persistent for all; +drop table t1; + +CREATE TABLE t1 (a longtext, FULLTEXT KEY (`a`)) ENGINE=MyISAM; +insert into t1 values (unhex('3E0D0A4141414142334E7A6143317963324541414141424977414141674541')); +analyze table t1 persistent for all; +drop table t1; + +CREATE TABLE t1 (f longtext NOT NULL, FULLTEXT KEY f (f)) ENGINE=MyISAM; +INSERT INTO t1 VALUES (REPEAT('a',27)),('foo'); +ANALYZE TABLE t1 PERSISTENT FOR ALL; +drop table t1; diff --git a/sql/sql_statistics.cc b/sql/sql_statistics.cc index d368145..2c1cccf 100644 --- a/sql/sql_statistics.cc +++ b/sql/sql_statistics.cc @@ -2367,6 +2367,9 @@ int collect_statistics_for_index(THD *thd, TABLE *table, uint index) DBUG_RETURN(rc); } + if (key_info->flags & HA_FULLTEXT) // skip if FULLTEXT index + DBUG_RETURN(rc); + table->key_read= 1; table->file->extra(HA_EXTRA_KEYREAD); Regards, Vicențiu

3 5

[Maria-developers] MDEV-7440 Build fails in libgroonga on Solaris
by Alexander Barkov 16 Jan '15

16 Jan '15

Hello Kentoku, Can you please have a look into this problem? https://mariadb.atlassian.net/browse/MDEV-7440 Can you fix please? Thanks.

1 0

Re: [Maria-developers] [Commits] Rev 4398: MDEV-7368 : SLES: Failed to start mysql.service: Unit
by Sergei Golubchik 15 Jan '15

15 Jan '15

Hi, Nirbhay! On Dec 23, Nirbhay Choubey wrote: > At lp:~maria-captains/maria/5.5 > > ------------------------------------------------------------ > revno: 4398 > revision-id: nirbhay(a)mariadb.com-20141224033207-8lh5i8d84qsjey79 > parent: elenst(a)montyprogram.com-20141223172123-pkfhjnv7oy1suxpa > committer: Nirbhay Choubey <nirbhay(a)mariadb.com> > branch nick: b7368-5.5 > timestamp: Tue 2014-12-23 22:32:07 -0500 > message: > MDEV-7368 : SLES: Failed to start mysql.service: Unit > mysql.service failed to load > > Added 'systemctl daemon-reload' command in postin and > postun rpm scripts. > === modified file 'support-files/rpm/server-postin.sh' > --- a/support-files/rpm/server-postin.sh 2013-11-20 11:18:46 +0000 > +++ b/support-files/rpm/server-postin.sh 2014-12-24 03:32:07 +0000 > @@ -1,6 +1,10 @@ > > # Make MySQL start/shutdown automatically when the machine does it. > if [ $1 = 1 ] ; then > + if [ -x /usr/bin/systemctl ] ; then > + /usr/bin/systemctl daemon-reload >/dev/null 2>&1 > + fi > + looks ok > if [ -x /sbin/chkconfig ] ; then > /sbin/chkconfig --add mysql > fi > @@ -51,30 +55,31 @@ > SETARGETDIR=/etc/selinux/targeted/src/policy > SEDOMPROG=$SETARGETDIR/domains/program > SECONPROG=$SETARGETDIR/file_contexts/program > +if [ -f /etc/redhat-release ] ; then > + if grep -q "Red Hat Enterprise Linux .. release 4" /etc/redhat-release \ > + || grep -q "CentOS release 4" /etc/redhat-release ; then I'd rather written if grep -Eqs '(Red Hat Enterprise Linux ..|CentOS) release 4' /etc/redhat-release; then or (which is more portable, but perhaps unnecessary so for an rpm postin script) if grep '$Red Hat Enterprise Linux ..\|CentOS$ release 4' /etc/redhat-release >/dev/null 2>&1; then but if you'd like you can push your variant. Regards, Sergei

2 1

Re: [Maria-developers] [Commits] Rev 4401: MDEV-7448 - mtr may leave stale mysqld in lp:maria/5.5
by Sergei Golubchik 14 Jan '15

14 Jan '15

Hi, Sergey! On Jan 14, Sergey Vojtovich wrote: > ------------------------------------------------------------ > revno: 4401 > revision-id: svoj(a)mariadb.org-20150114135038-jss2vvpkhihysfmj > parent: sergii(a)pisem.net-20150110130746-xlrofqddt4j8xkw6 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 5.5 > timestamp: Wed 2015-01-14 17:50:38 +0400 > message: > MDEV-7448 - mtr may leave stale mysqld > > Let mtr handle SIGHUP the same way as SIGINT. This solves stale mysqld processes > left after broken/closed ssh connection. Looks ok. But please make sure that all tests pass before pushing it. mysqld uses SIGHUP to print various information to the error log. If there's a test that actually tests that by sending SIGHUP - your change might've broken it. Regards, Sergei

1 0

[Maria-developers] MySQL bug #68557 "sjis changes in 5.5 break date parsing" fixed in MariaDB >= 5.5.38
by Hartmut Holzgraefe 13 Jan '15

13 Jan '15

Any idea how that happened, as I can't find anything that seems to be related in the 5.5.38 changelog? http://bugs.mysql.com/bug.php?id=68557 -- Hartmut Holzgraefe, Principal Support Engineer (EMEA) MariaDB Corporation | http://www.mariadb.com/

2 1

[Maria-developers] Buildbot slow upload
by Kristian Nielsen 13 Jan '15

13 Jan '15

Hi Serg, Daniel, Some time ago there was discussion of slow Buildbot upload speed (sorry, don't have the original mail thread to reply to). I found some new information on this. Yesterday I had to do a custom build, but was not able to because my tarbake upload kept getting stuck. So I had to investigate. Turns out the real problem was that hasky:/media/backup/archive/pack was full. And apparently, the error handling in Buildbot upload step is broken, so the error is not caught properly but just causes the upload to hang. I have noticed this before, that upload gets stuck. I wonder if full disk on hasky has caused this before. Daniel: Do we have something that removes old builds from /archive/pack? I did not see anything in root or buildbot crontabs? With respect to slow uploads, while trying various things, one thing I did was increase the upload blocksize in Buildbot config. Now I see tarbake upload (50MB) taking just 1 minute, which seems fair (probably it can be more if the network or buildbot is busy). So I'm wondering if the slow upload that was seen was in fact a combination of too low default blocksize, and uploads occasionally getting _completely_ stuck? It's of course still fine to move to using rsync, as Serg proposed earlier; just wanted to mention what I found. And in either case, we still need a solution to prevent /archive/pack from running full in the future... - Kristian.

2 1

[Maria-developers] MDEV-7288 USER/ROLE: CREATE OR REPLACE, CREATE IF NOT EXISTS, DROP IF EXISTS
by Alexander Barkov 09 Jan '15

09 Jan '15

Hi Sergei, Please have a look into the patch for MDEV-7288. It consists of the USER/ROLE related part of the big GSoC patch from Sriram that you reviewed earlier, with review suggestions addressed. You earlier had the following suggestions/questions: >> diff --git a/sql/sql_acl.cc b/sql/sql_acl.cc >> index 5f077c2..1e156be 100644 >> --- a/sql/sql_acl.cc >> +++ b/sql/sql_acl.cc >> ... >> @@ -9215,10 +9252,26 @@ bool mysql_create_user(THD *thd, List <LEX_USER> &lis >> */ >> if (handle_grant_data(tables, 0, user_name, NULL)) >> { >> - append_user(thd, &wrong_users, user_name); >> - >> - result= TRUE; >> - continue; >> + if (thd->lex->is_create_or_replace()) >> + { >> + if (handle_grant_data(tables, 1, user_name, NULL) <= 0) >> + { >> + append_user(thd, &wrong_users, user_name); > > I don't understand that at all. If user creation failed, you try to drop > it, but don't create a user again, after the old one was dropped. How > could it work? > Yeah, it looks confusing, because handle_grant_data() can do different things depending on the arguments. The call for handle_grant_data(tables, 0, user_name, NULL) does not create any users. It only checks if the user already exists. The call for handle_grant_data(tables, 1, user_name, NULL) drops the user. And the call for replace_user_table() after this block is the one that actually creates the user. I have added some comments in the new code, to make it clearer. >>> @@ -9268,7 +9321,7 @@ bool mysql_create_user(THD *thd, List <LEX_USER> &list, >>> (handle_as_role) ? "CREATE ROLE" : "CREATE USER", >>> wrong_users.c_ptr_safe()); >>> >>> - if (some_users_created) >>> + if (some_users_created || !result) >> >> What is that for? To binlog CREATE IF NOT EXISTS? Yes, that was the reason. I renamed "some_users_created" to "binlog" and changed the logic slightly to set "binlog" to true under the block checking lex->create_info.if_not_exists(), instead of this additional test for "result". So now it should be easier to read: if (binlog) result |= write_bin_log(thd, FALSE, thd->query(), thd->query_length()); Thanks.

3 3

[Maria-developers] prefix matching for command-line options - how to disable?
by Sergei Golubchik 06 Jan '15

06 Jan '15

Hi, We've got a bit of a problem with command-line option prefixes in the test suite. As you know, in MariaDB a command line option (the same applies to my.cnf too) doesn't need to be specified completely, it's sufficient to use an unambigous prefix of it. For example, 'net-retry-count=5' in my.cnf can be abbreviated as 'net-ret=5'. But not as 'net-re=5', the latter will fail with an error mysqld: ambiguous option '--net-re' (net_read_timeout, net_retry_count) This is the historical behavior of MySQL since about 2002. But it creates problems for plugins. InnoDB is particularly badly affected - it has lots of I_S plugins that are prefixes of other I_S plugins. For example: INNODB_BUFFER_PAGE INNODB_BUFFER_PAGE_LRU ... INNODB_CMP INNODB_CMPMEM INNODB_CMPMEM_RESET INNODB_CMP_PER_INDEX INNODB_CMP_PER_INDEX_RESET INNODB_CMP_RESET ... INNODB_SYS_FOREIGN INNODB_SYS_FOREIGN_COLS ... INNODB_SYS_TABLES INNODB_SYS_TABLESPACES INNODB_SYS_TABLESTATS also, it has plugins that match other plugin variables, like INNODB_LOCKS and innodb_locks_unsafe_for_binlog. As a result one cannot see or predict what these options will do. For example --enable-innodb-buffer-page might enable INNODB_BUFFER_PAGE plugin. Or it might enable INNODB_BUFFER_PAGE_LRU plugin - depending on which plugin will loaded (and will see and consume the option) first. And one cannot possibly disable INNODB_BUFFER_PAGE plugin, if the server will try to load INNODB_BUFFER_PAGE_LRU first. I see few solutions for this issue: 1. support a special suffix to option names that will disable prefix matching, for example: mysqld --enable-innodb-buffer-page! this will not match innodb-buffer-page-lru, so it's unambiguous. Of course, any dedicated character can be used, --enable-innodb-buffer-page$, or even a prefix --enable-exact-innodb-buffer-page, --enable-strict-innodb-buffer-page 2. add a special command-line option to disable prefix matching: mysqld --disable-getopt-prefix-matching --enable-innodb-buffer-page 3. do like mysql-5.7 is doing and completely remove support for prefix matching. In 5.7 one has to specified option names in full. I suspect that this is likely to break many exising setups. Opinions? Anybody cares about this obscure issue at all? Regards, Sergei

3 3

[Maria-developers] Buildslave terrier2 disabled due to KVM problems, needs fixing to be re-enabled
by Kristian Nielsen 05 Jan '15

05 Jan '15

Jani, Daniel, I discovered that our buildslave terrier2 seems to have problems with its KVM. Builds are incredibly slow, and it appears that it is not using KVM at all. So it is apparently using just QEMU, and soft-emulating the CPU. This is too slow to be useful, I think. So for now, I've removed all usage of terrier2 from the Buildbot config. If one of you wants to fix terrier2 to be useful again, then the patch to revert (to re-enable terrier2) is this: revid:knielsen@knielsen-hq.org-20150105182519-7492d95v1blss8jf http://bazaar.launchpad.net/~maria-captains/mariadb-tools/trunk/revision/kn… Before re-enabling, KVM needs to be somehow fixed to work (eg. something like `time perl -le 'for (0..100000000){}'` needs to be tested to run at about the same speed inside a guest VM as directly on the host). Maybe it would be best to upgrade the OS on the machine, maybe to the same as we have on bb01/bb02? We have had a number of problems on terrier2 with various KVM issues over the time. - Kristian.

1 0

[Maria-developers] New buildbot? - debpkg101-sid from Otto's github
by Otto Kekäläinen 02 Jan '15

02 Jan '15

Hello Daniel! Thanks for setting up the current buildbots that build my branches of 5.5 and 10.0. Could you now please also set up one the build from my 10.1 branch as I have started working on it too? The buildbot should be a copy of http://buildbot.askmonty.org/buildbot/builders/debpkg-sid/ but so that it pulls in the branch 10.1 from the repository https://github.com/ottok/mariadb Thanks! 2014-09-01 11:25 GMT+03:00 Otto Kekäläinen <otto(a)seravo.fi>: > Hello! > > 2014-03-14 12:59 GMT+02:00 Kristian Nielsen <knielsen(a)knielsen-hq.org>: >> You can see the two builders here: >> >> http://buildbot.askmonty.org/buildbot/builders/debpkg-trusty >> http://buildbot.askmonty.org/buildbot/builders/debpkg-sid/ > > Thanks Kristian and Daniel for creating these in March. > > The trusty build testing is less relevant nowadays while testing 10.0 > has become relevant. > > Could you please delete the buildbot debpkg-trusty and create a copy > of debpkg-sid called debpkg-10.0-sid which would be exactly like > debpkg-sid, but follow commits in this repo: > https://github.com/ottok/mariadb-10.0 ? > > Thanks! > > PS. I already configured the webhook in Github to be called in the > mariadb-10.0 repo as it is done in the mariadb-5.5 repo.

1 2

Re: [Maria-developers] [Commits] b2f45ba: MDEV-7324 - Lock-free hash for table definition cache
by Sergei Golubchik 28 Dec '14

28 Dec '14

Hi, Sergey! On Dec 22, svoj(a)mariadb.org wrote: > revision-id: b2f45ba8280c97f8dc8b1adb3234cf2e28e48982 > parent(s): 4a469b0ed68bb341e1a1e1ba540a45a3b6ff3ef9 > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2014-12-22 19:56:48 +0400 > message: > > MDEV-7324 - Lock-free hash for table definition cache That looked good to me. I didn't find anything obviously wrong. There are mostly questions below. Some are style suggestions (like, renaming of methods, etc). The only place I'm not sure about is early lf_hash_search_unpin() in tdc_acquire_share(). > diff --git a/mysys/lf_hash.c b/mysys/lf_hash.c > --- a/mysys/lf_hash.c > +++ b/mysys/lf_hash.c > @@ -122,7 +122,7 @@ static int lfind(LF_SLIST * volatile *head, CHARSET_INFO *cs, uint32 hashnr, > { > if (unlikely(callback)) > { > - if (callback(cursor->curr + 1, (void*)key)) > + if (cur_hashnr & 1 && callback(cursor->curr + 1, (void*)key)) Sorry for this. > return 1; > } > else if (cur_hashnr >= hashnr) > diff --git a/sql/sql_class.cc b/sql/sql_class.cc > --- a/sql/sql_class.cc > +++ b/sql/sql_class.cc > @@ -912,17 +912,17 @@ bool Drop_table_error_handler::handle_condition(THD *thd, > #endif /* defined(ENABLED_DEBUG_SYNC) */ > wait_for_commit_ptr(0), > main_da(0, false, false), > - m_stmt_da(&main_da) > + m_stmt_da(&main_da), > #ifdef WITH_WSREP > - , > wsrep_applier(is_wsrep_applier), > wsrep_applier_closing(false), > wsrep_client_thread(false), > wsrep_apply_toi(false), > wsrep_po_handle(WSREP_PO_INITIALIZER), > wsrep_po_cnt(0), > - wsrep_apply_format(0) > + wsrep_apply_format(0), > #endif > + tdc_hash_pins(0) Eh... I actually wanted wsrep related data to the at the end of the THD. Just to keep the rest of the THD layout identical for wsrep and non-wsrep builds. In case a wsrep-enabled plugin is loaded into non-wsrep server or visa versa. That was an early change, though, I've done more cleanups after that and moved wsrep into a service, so perhaps (even, most probably) this is not needed anymore. But I'd better stay on the safe side and keep wsrep at the end of THD. > { > ulong tmp; > > @@ -1698,6 +1698,8 @@ void THD::cleanup(void) > > free_root(&main_mem_root, MYF(0)); > main_da.free_memory(); > + if (tdc_hash_pins) > + lf_hash_put_pins(tdc_hash_pins); no tdc_hash_pins=0 here? > if (status_var.memory_used != 0) > { > DBUG_PRINT("error", ("memory_used: %lld", status_var.memory_used)); > diff --git a/sql/table.h b/sql/table.h > --- a/sql/table.h > +++ b/sql/table.h > @@ -611,32 +612,7 @@ struct TABLE_SHARE > mysql_mutex_t LOCK_ha_data; /* To protect access to ha_data */ > mysql_mutex_t LOCK_share; /* To protect TABLE_SHARE */ > > - typedef I_P_List <TABLE, TABLE_share> TABLE_list; > - typedef I_P_List <TABLE, All_share_tables> All_share_tables_list; > - struct > - { > - /** > - Protects ref_count, m_flush_tickets, all_tables, free_tables, flushed, > - all_tables_refs. > - */ > - mysql_mutex_t LOCK_table_share; > - mysql_cond_t COND_release; > - TABLE_SHARE *next, **prev; /* Link to unused shares */ > - uint ref_count; /* How many TABLE objects uses this */ > - uint all_tables_refs; /* Number of refs to all_tables */ > - /** > - List of tickets representing threads waiting for the share to be flushed. > - */ > - Wait_for_flush_list m_flush_tickets; > - /* > - Doubly-linked (back-linked) lists of used and unused TABLE objects > - for this share. > - */ > - All_share_tables_list all_tables; > - TABLE_list free_tables; > - ulong version; > - bool flushed; > - } tdc; > + TDC_element *tdc; Hmm. Ok. I think I like this change - TABLE_SHARE now contains table metadata and everything related to tdc is moved to TDC_element. Right? > > LEX_CUSTRING tabledef_version; > > diff --git a/sql/table.cc b/sql/table.cc > --- a/sql/table.cc > +++ b/sql/table.cc > @@ -3845,11 +3844,11 @@ bool TABLE_SHARE::visit_subgraph(Wait_for_flush *wait_for_flush, > because we won't try to acquire tdc.LOCK_table_share while > holding a write-lock on MDL_lock::m_rwlock. > */ > - mysql_mutex_lock(&tdc.LOCK_table_share); > - tdc.all_tables_refs++; > - mysql_mutex_unlock(&tdc.LOCK_table_share); > + mysql_mutex_lock(&tdc->LOCK_table_share); > + tdc->all_tables_refs++; > + mysql_mutex_unlock(&tdc->LOCK_table_share); Do I understand it correctly that a TDC_element can never be deleted from LF_HASH as long as at least one TABLE_SHARE references it? > > - All_share_tables_list::Iterator tables_it(tdc.all_tables); > + TDC_element::All_share_tables_list::Iterator tables_it(tdc->all_tables); > > /* > In case of multiple searches running in parallel, avoid going > @@ -3946,21 +3945,10 @@ bool TABLE_SHARE::wait_for_old_version(THD *thd, struct timespec *abstime, > > mdl_context->done_waiting_for(); > > - mysql_mutex_lock(&tdc.LOCK_table_share); > - > - tdc.m_flush_tickets.remove(&ticket); > - > - if (tdc.m_flush_tickets.is_empty() && tdc.ref_count == 0) > - { > - /* > - If our thread was the last one using the share, > - we must destroy it here. > - */ > - mysql_mutex_unlock(&tdc.LOCK_table_share); > - destroy(); > - } > - else > - mysql_mutex_unlock(&tdc.LOCK_table_share); > + mysql_mutex_lock(&tdc->LOCK_table_share); > + tdc->m_flush_tickets.remove(&ticket); > + mysql_cond_broadcast(&tdc->COND_release); > + mysql_mutex_unlock(&tdc->LOCK_table_share); Why is the logic here different? > > > /* > diff --git a/sql/table_cache.h b/sql/table_cache.h > --- a/sql/table_cache.h > +++ b/sql/table_cache.h > @@ -16,6 +16,171 @@ > Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */ > > > +extern PSI_mutex_key key_TABLE_SHARE_LOCK_table_share; > +extern PSI_cond_key key_TABLE_SHARE_COND_release; > + > +class TDC_element > +{ > +public: > + uchar m_key[NAME_LEN + 1 + NAME_LEN + 1]; > + uint m_key_length; > + ulong version; > + bool flushed; > + TABLE_SHARE *share; > + > + typedef I_P_List <TABLE, TABLE_share> TABLE_list; > + typedef I_P_List <TABLE, All_share_tables> All_share_tables_list; > + /** > + Protects ref_count, m_flush_tickets, all_tables, free_tables, flushed, > + all_tables_refs. > + */ > + mysql_mutex_t LOCK_table_share; > + mysql_cond_t COND_release; > + TDC_element *next, **prev; /* Link to unused shares */ > + uint ref_count; /* How many TABLE objects uses this */ > + uint all_tables_refs; /* Number of refs to all_tables */ > + /** > + List of tickets representing threads waiting for the share to be flushed. > + */ > + Wait_for_flush_list m_flush_tickets; > + /* > + Doubly-linked (back-linked) lists of used and unused TABLE objects > + for this share. > + */ > + All_share_tables_list all_tables; > + TABLE_list free_tables; > + > + TDC_element() {} > + > + TDC_element(const char *key, uint key_length) : m_key_length(key_length) > + { > + memcpy(m_key, key, key_length); > + } > + > + > + void dbug_release_assertions() Somewhat confusing name. If I hadn't see the code below I'd thought this function releases some assertions (how? why?) may be assert_correctness_at_release() would be less ambiguous? > + { > + DBUG_ASSERT(share == 0); > + DBUG_ASSERT(ref_count == 0); > + DBUG_ASSERT(m_flush_tickets.is_empty()); > + DBUG_ASSERT(all_tables.is_empty()); > + DBUG_ASSERT(free_tables.is_empty()); > + DBUG_ASSERT(all_tables_refs == 0); > + DBUG_ASSERT(next == 0); > + DBUG_ASSERT(prev == 0); > + } > + > + > + /** > + Acquire TABLE object from table cache. > + > + @pre share must be protected against removal. > + > + Acquired object cannot be evicted or acquired again. > + > + While locked: > + - pop object from TABLE_SHARE::tdc.free_tables > + > + While unlocked: > + - mark object used by thd "While locked" / "While unlocked" - what does that mean? > + > + @return TABLE object, or NULL if no unused objects. > + */ > + > + TABLE *acquire_table(THD *thd) > + { > + TABLE *table; > + > + mysql_mutex_lock(&LOCK_table_share); > + table= free_tables.pop_front(); > + if (table) > + { > + DBUG_ASSERT(!table->in_use); lots of asserts - good! > + table->in_use= thd; > + /* The ex-unused table must be fully functional. */ > + DBUG_ASSERT(table->db_stat && table->file); > + /* The children must be detached from the table. */ > + DBUG_ASSERT(!table->file->extra(HA_EXTRA_IS_ATTACHED_CHILDREN)); I don't even think of asking why it's HA_EXTRA_IS_ATTACHED_CHILDREN and not HA_EXTRA_HAS_ATTACHED_CHILDREN or, at least, HA_EXTRA_CHILDREN_ARE_ATTACHED. Ah, whatever... Horrors of the English language hidden in old MySQL code :) > + } > + mysql_mutex_unlock(&LOCK_table_share); > + return table; > + } > + > + > + /** > + Get last element of free_tables. > + */ > + > + TABLE *free_tables_back() Eh, I've thought for a few seconds what could it mean to "free tables back" until I read the function comment. You see, "free" is so often used as a verb that one absolutely doesn't see it that here it means "free_tables' back" even though it's also a common naming pattern (xxx_back for the list xxx). Rename the method, may be? > + { > + TABLE_list::Iterator it(share->tdc->free_tables); > + TABLE *entry, *last= 0; > + while ((entry= it++)) > + last= entry; > + return last; > + } > + > + > + /** > + Wait for MDL deadlock detector to complete traversing tdc.all_tables. > + > + Must be called before updating TABLE_SHARE::tdc.all_tables. > + */ > + > + void wait_for_mdl_deadlock_detector() > + { > + while (all_tables_refs) > + mysql_cond_wait(&COND_release, &LOCK_table_share); > + } > + > + > + /** > + Prepeare table share for use with table definition cache. > + */ > + > + static void lf_alloc_constructor(uchar *arg) > + { > + TDC_element *element= (TDC_element*) (arg + LF_HASH_OVERHEAD); > + DBUG_ENTER("tdc_init_share"); intentionally? or forgot to update DBUG_ENTER? > + mysql_mutex_init(key_TABLE_SHARE_LOCK_table_share, > + &element->LOCK_table_share, MY_MUTEX_INIT_FAST); > + mysql_cond_init(key_TABLE_SHARE_COND_release, &element->COND_release, 0); > + element->m_flush_tickets.empty(); > + element->all_tables.empty(); > + element->free_tables.empty(); > + element->all_tables_refs= 0; > + element->share= 0; > + element->ref_count= 0; > + element->next= 0; > + element->prev= 0; > + DBUG_VOID_RETURN; > + } > + > + > + /** > + Release table definition cache specific resources of table share. > + */ > + > + static void lf_alloc_destructor(uchar *arg) > + { > + TDC_element *element= (TDC_element*) (arg + LF_HASH_OVERHEAD); > + DBUG_ENTER("tdc_deinit_share"); intentionally? or forgot to update DBUG_ENTER? > + element->dbug_release_assertions(); > + mysql_cond_destroy(&element->COND_release); > + mysql_mutex_destroy(&element->LOCK_table_share); > + DBUG_VOID_RETURN; > + } > + > + > + static uchar *key(const TDC_element *element, size_t *length, > + my_bool not_used __attribute__((unused))) > + { > + *length= element->m_key_length; > + return (uchar*) element->m_key; > + } > +}; > + > + > enum enum_tdc_remove_table_type > { > TDC_RT_REMOVE_ALL, > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -270,58 +270,75 @@ uint get_table_def_key(const TABLE_LIST *table_list, const char **key) > # Pointer to list of names of open tables. > */ > > -OPEN_TABLE_LIST *list_open_tables(THD *thd, const char *db, const char *wild) > +struct list_open_tables_arg > { > + THD *thd; > + const char *db; > + const char *wild; > TABLE_LIST table_list; > OPEN_TABLE_LIST **start_list, *open_list; > - TABLE_SHARE *share; > - TDC_iterator tdc_it; > - DBUG_ENTER("list_open_tables"); > +}; > > - bzero((char*) &table_list,sizeof(table_list)); > - start_list= &open_list; > - open_list=0; > > - tdc_it.init(); > - while ((share= tdc_it.next())) > +static my_bool list_open_tables_callback(TDC_element *element, > + list_open_tables_arg *arg) > { > - if (db && my_strcasecmp(system_charset_info, db, share->db.str)) > - continue; > - if (wild && wild_compare(share->table_name.str, wild, 0)) > - continue; > + char *db= (char*) element->m_key; > + char *table_name= (char*) element->m_key + strlen((char*) element->m_key) + 1; > > + if (arg->db && my_strcasecmp(system_charset_info, arg->db, db)) > + return FALSE; > + if (arg->wild && wild_compare(table_name, arg->wild, 0)) > + return FALSE; > + > /* Check if user has SELECT privilege for any column in the table */ > - table_list.db= share->db.str; > - table_list.table_name= share->table_name.str; > - table_list.grant.privilege=0; > + arg->table_list.db= db; > + arg->table_list.table_name= table_name; > + arg->table_list.grant.privilege= 0; > > - if (check_table_access(thd,SELECT_ACL,&table_list, TRUE, 1, TRUE)) > - continue; > + if (check_table_access(arg->thd, SELECT_ACL, &arg->table_list, TRUE, 1, TRUE)) > + return FALSE; > > - if (!(*start_list = (OPEN_TABLE_LIST *) > - sql_alloc(sizeof(**start_list)+share->table_cache_key.length))) > - { > - open_list=0; // Out of memory > - break; > - } > + if (!(*arg->start_list= (OPEN_TABLE_LIST *) sql_alloc( when you know the thd, prefer thd->alloc() over sql_alloc() (which is, basically, current_thd->alloc()) > + sizeof(**arg->start_list) + element->m_key_length))) > + return TRUE; > + > - strmov((*start_list)->table= > - strmov(((*start_list)->db= (char*) ((*start_list)+1)), > - share->db.str)+1, > - share->table_name.str); > + strmov((*arg->start_list)->table= > + strmov(((*arg->start_list)->db= (char*) ((*arg->start_list) + 1)), > + db) + 1, table_name); > - (*start_list)->in_use= 0; > + (*arg->start_list)->in_use= 0; > - mysql_mutex_lock(&share->tdc.LOCK_table_share); > - TABLE_SHARE::All_share_tables_list::Iterator it(share->tdc.all_tables); > + > + mysql_mutex_lock(&element->LOCK_table_share); > + TDC_element::All_share_tables_list::Iterator it(element->all_tables); > TABLE *table; > while ((table= it++)) > if (table->in_use) > - ++(*start_list)->in_use; > + ++(*arg->start_list)->in_use; > - mysql_mutex_unlock(&share->tdc.LOCK_table_share); > + mysql_mutex_unlock(&element->LOCK_table_share); > - (*start_list)->locked= 0; /* Obsolete. */ > + (*arg->start_list)->locked= 0; /* Obsolete. */ > - start_list= &(*start_list)->next; > + arg->start_list= &(*arg->start_list)->next; > - *start_list=0; > + *arg->start_list= 0; > + return FALSE; > } > - tdc_it.deinit(); > - DBUG_RETURN(open_list); > + > + > +OPEN_TABLE_LIST *list_open_tables(THD *thd, const char *db, const char *wild) > +{ > + list_open_tables_arg argument; > + DBUG_ENTER("list_open_tables"); > + > + argument.thd= thd; > + argument.db= db; > + argument.wild= wild; > + bzero((char*) &argument.table_list, sizeof(argument.table_list)); > + argument.start_list= &argument.open_list; > + argument.open_list= 0; > + > + if (tdc_iterate(thd, (my_hash_walk_action) list_open_tables_callback, > + &argument, true)) > + DBUG_RETURN(0); > + > + DBUG_RETURN(argument.open_list); > } > > /***************************************************************************** > diff --git a/sql/table_cache.cc b/sql/table_cache.cc > --- a/sql/table_cache.cc > +++ b/sql/table_cache.cc > @@ -72,32 +76,23 @@ > /** > Protects unused shares list. > > - TABLE_SHARE::tdc.prev > - TABLE_SHARE::tdc.next > - oldest_unused_share > - end_of_unused_share > + TDC_element::prev > + TDC_element::next > + unused_shares > */ > > static mysql_mutex_t LOCK_unused_shares; > -static mysql_rwlock_t LOCK_tdc; /**< Protects tdc_hash. */ > my_atomic_rwlock_t LOCK_tdc_atomics; /**< Protects tdc_version. */ > > #ifdef HAVE_PSI_INTERFACE > -static PSI_mutex_key key_LOCK_unused_shares, key_TABLE_SHARE_LOCK_table_share; > +PSI_mutex_key key_LOCK_unused_shares, key_TABLE_SHARE_LOCK_table_share; really? not static? > static PSI_mutex_info all_tc_mutexes[]= > { > { &key_LOCK_unused_shares, "LOCK_unused_shares", PSI_FLAG_GLOBAL }, > { &key_TABLE_SHARE_LOCK_table_share, "TABLE_SHARE::tdc.LOCK_table_share", 0 } > }; > > -static PSI_rwlock_key key_rwlock_LOCK_tdc; > -static PSI_rwlock_info all_tc_rwlocks[]= > -{ > - { &key_rwlock_LOCK_tdc, "LOCK_tdc", PSI_FLAG_GLOBAL } > -}; > - > - > -static PSI_cond_key key_TABLE_SHARE_COND_release; > +PSI_cond_key key_TABLE_SHARE_COND_release; and here > static PSI_cond_info all_tc_conds[]= > { > { &key_TABLE_SHARE_COND_release, "TABLE_SHARE::tdc.COND_release", 0 } > @@ -265,85 +269,41 @@ void tc_add_table(THD *thd, TABLE *table) > > if (need_purge) > { > - TABLE_SHARE *purge_share= 0; > - TABLE_SHARE *share; > - TABLE *entry; > - ulonglong UNINIT_VAR(purge_time); > - TDC_iterator tdc_it; > + tc_add_table_arg argument; > + argument.purge_time= ULONGLONG_MAX; > + tdc_iterate(thd, (my_hash_walk_action) tc_add_table_callback, &argument); > > - tdc_it.init(); > - while ((share= tdc_it.next())) > + if (argument.purge_time != ULONGLONG_MAX) > { > - mysql_mutex_lock(&share->tdc.LOCK_table_share); > - if ((entry= tc_free_tables_back(share)) && > - (!purge_share || entry->tc_time < purge_time)) > + TDC_element *element= (TDC_element*) lf_hash_search(&tdc_hash, > + thd->tdc_hash_pins, > + argument.key, > + argument.key_length); > + if (element) > { > - purge_share= share; > - purge_time= entry->tc_time; > - } > - mysql_mutex_unlock(&share->tdc.LOCK_table_share); > - } > + TABLE *entry; > + mysql_mutex_lock(&element->LOCK_table_share); > + lf_hash_search_unpin(thd->tdc_hash_pins); > + element->wait_for_mdl_deadlock_detector(); you wait under a mutex? hmm... > > - if (purge_share) > - { > - mysql_mutex_lock(&purge_share->tdc.LOCK_table_share); > - tc_wait_for_mdl_deadlock_detector(purge_share); > - tdc_it.deinit(); > /* > It may happen that oldest table was acquired meanwhile. In this case > just go ahead, number of objects in table cache will normalize > eventually. > */ > - if ((entry= tc_free_tables_back(purge_share)) && > - entry->tc_time == purge_time) > + if ((entry= element->free_tables_back()) && > + entry->tc_time == argument.purge_time) > { > - entry->s->tdc.free_tables.remove(entry); > + element->free_tables.remove(entry); > tc_remove_table(entry); > - mysql_mutex_unlock(&purge_share->tdc.LOCK_table_share); > + mysql_mutex_unlock(&element->LOCK_table_share); > intern_close_table(entry); > } > else > - mysql_mutex_unlock(&purge_share->tdc.LOCK_table_share); > + mysql_mutex_unlock(&element->LOCK_table_share); > } > - else > - tdc_it.deinit(); > } > } > - > - > -/** > - Acquire TABLE object from table cache. > - > - @pre share must be protected against removal. > - > - Acquired object cannot be evicted or acquired again. > - > - While locked: > - - pop object from TABLE_SHARE::tdc.free_tables > - > - While unlocked: > - - mark object used by thd > - > - @return TABLE object, or NULL if no unused objects. > -*/ > - > -static TABLE *tc_acquire_table(THD *thd, TABLE_SHARE *share) > -{ > - TABLE *table; > - > - mysql_mutex_lock(&share->tdc.LOCK_table_share); > - table= share->tdc.free_tables.pop_front(); > - if (table) > - { > - DBUG_ASSERT(!table->in_use); > - table->in_use= thd; > - /* The ex-unused table must be fully functional. */ > - DBUG_ASSERT(table->db_stat && table->file); > - /* The children must be detached from the table. */ > - DBUG_ASSERT(!table->file->extra(HA_EXTRA_IS_ATTACHED_CHILDREN)); > - } ok, you've moved it to table_cache.h... > - mysql_mutex_unlock(&share->tdc.LOCK_table_share); > - return table; > } > > > @@ -397,88 +357,76 @@ bool tc_release_table(TABLE *table) > */ > table->in_use= 0; > /* Add table to the list of unused TABLE objects for this share. */ > - table->s->tdc.free_tables.push_front(table); > - mysql_mutex_unlock(&table->s->tdc.LOCK_table_share); > + table->s->tdc->free_tables.push_front(table); > + mysql_mutex_unlock(&table->s->tdc->LOCK_table_share); > return false; > > purge: > - tc_wait_for_mdl_deadlock_detector(table->s); > + table->s->tdc->wait_for_mdl_deadlock_detector(); > tc_remove_table(table); > - mysql_mutex_unlock(&table->s->tdc.LOCK_table_share); > + mysql_mutex_unlock(&table->s->tdc->LOCK_table_share); > table->in_use= 0; > intern_close_table(table); > return true; > } > > > -extern "C" uchar *tdc_key(const uchar *record, size_t *length, > - my_bool not_used __attribute__((unused))) > -{ > - TABLE_SHARE *entry= (TABLE_SHARE*) record; > - *length= entry->table_cache_key.length; > - return (uchar*) entry->table_cache_key.str; > -} > - > - > /** > Delete share from hash and free share object. > - > - @return > - @retval 0 Success > - @retval 1 Share is referenced > */ > > -static int tdc_delete_share_from_hash(TABLE_SHARE *share) > +static void tdc_delete_share_from_hash(TDC_element *element) > { > + THD *thd= current_thd; > + LF_PINS *pins; > + TABLE_SHARE *share; > DBUG_ENTER("tdc_delete_share_from_hash"); > - mysql_rwlock_wrlock(&LOCK_tdc); > - mysql_mutex_lock(&share->tdc.LOCK_table_share); > - if (--share->tdc.ref_count) > - { > - mysql_cond_broadcast(&share->tdc.COND_release); > - mysql_mutex_unlock(&share->tdc.LOCK_table_share); > - mysql_rwlock_unlock(&LOCK_tdc); > - DBUG_RETURN(1); > - } > - my_hash_delete(&tdc_hash, (uchar*) share); > + > + share= element->share; > + DBUG_ASSERT(share); > + element->share= 0; > /* Notify PFS early, while still locked. */ "while still locked" what? > PSI_CALL_release_table_share(share->m_psi); > share->m_psi= 0; > - mysql_rwlock_unlock(&LOCK_tdc); > > - if (share->tdc.m_flush_tickets.is_empty()) > + if (!element->m_flush_tickets.is_empty()) > { > - /* No threads are waiting for this share to be flushed, destroy it. */ > - mysql_mutex_unlock(&share->tdc.LOCK_table_share); > - free_table_share(share); > - } > - else > - { > - Wait_for_flush_list::Iterator it(share->tdc.m_flush_tickets); > + Wait_for_flush_list::Iterator it(element->m_flush_tickets); > Wait_for_flush *ticket; > while ((ticket= it++)) > (void) ticket->get_ctx()->m_wait.set_status(MDL_wait::GRANTED); > - /* > - If there are threads waiting for this share to be flushed, > - the last one to receive the notification will destroy the > - share. At this point the share is removed from the table > - definition cache, so is OK to proceed here without waiting > - for this thread to do the work. > - */ > - mysql_mutex_unlock(&share->tdc.LOCK_table_share); > + > + do > + { > + mysql_cond_wait(&element->COND_release, &element->LOCK_table_share); > + } while (!element->m_flush_tickets.is_empty()); > } > - DBUG_RETURN(0); > + > + mysql_mutex_unlock(&element->LOCK_table_share); Ok, so the caller must lock LOCK_table_share? please put mysql_mutex_assert_owner() in the beginning of the function > + > + if (thd) When thd is NULL here? On shutdown? > + { > + fix_thd_pins(thd); > + pins= thd->tdc_hash_pins; > } > + else > + pins= lf_hash_get_pins(&tdc_hash); > > + DBUG_ASSERT(pins); // What can we do about it? > + element->dbug_release_assertions(); > + lf_hash_delete(&tdc_hash, pins, element->m_key, element->m_key_length); > + if (!thd) > + lf_hash_put_pins(pins); > + free_table_share(share); > + DBUG_VOID_RETURN; > +} > > + > /** > Initialize table definition cache. > - > - @retval 0 Success > - @retval !0 Error > */ > > -int tdc_init(void) > +void tdc_init(void) > { > DBUG_ENTER("tdc_init"); > #ifdef HAVE_PSI_INTERFACE > @@ -642,26 +548,34 @@ void tdc_deinit_share(TABLE_SHARE *share) > Caller is expected to unlock table share with tdc_unlock_share(). > > @retval 0 Share not found > - @retval !0 Pointer to locked table share > + @retval MY_ERRPTR OOM > + @retval ptr Pointer to locked table share > */ > > -TABLE_SHARE *tdc_lock_share(const char *db, const char *table_name) > +TDC_element *tdc_lock_share(THD *thd, const char *db, const char *table_name) > { > + TDC_element *element; > char key[MAX_DBKEY_LENGTH]; > - uint key_length; > > DBUG_ENTER("tdc_lock_share"); > - key_length= tdc_create_key(key, db, table_name); > + if (fix_thd_pins(thd)) > + DBUG_RETURN((TDC_element*) MY_ERRPTR); I suppose you can do it in THD:init, if you'd like. A THD almost always will need to access table definition cache, so it'll need these pins. Then there will be no need to return MY_ERRPTR here and handle it in the caller > > - mysql_rwlock_rdlock(&LOCK_tdc); > - TABLE_SHARE* share= (TABLE_SHARE*) my_hash_search(&tdc_hash, > - (uchar*) key, key_length); > - if (share && !share->error) > - mysql_mutex_lock(&share->tdc.LOCK_table_share); > - else > - share= 0; > - mysql_rwlock_unlock(&LOCK_tdc); > - DBUG_RETURN(share); > + element= (TDC_element *) lf_hash_search(&tdc_hash, thd->tdc_hash_pins, > + (uchar*) key, > + tdc_create_key(key, db, table_name)); > + if (element) > + { > + mysql_mutex_lock(&element->LOCK_table_share); > + if (!element->share || element->share->error) > + { > + mysql_mutex_unlock(&element->LOCK_table_share); > + element= 0; > + } > + lf_hash_search_unpin(thd->tdc_hash_pins); > + } > + > + DBUG_RETURN(element); Looks good to me > } > > > @@ -702,60 +616,60 @@ TABLE_SHARE *tdc_acquire_share(THD *thd, const char *db, const char *table_name, > TABLE **out_table) > { > TABLE_SHARE *share; > + TDC_element *element; > bool was_unused; > DBUG_ENTER("tdc_acquire_share"); > > - mysql_rwlock_rdlock(&LOCK_tdc); > - share= (TABLE_SHARE*) my_hash_search_using_hash_value(&tdc_hash, hash_value, > - (uchar*) key, > - key_length); > - if (!share) > + if (fix_thd_pins(thd)) > + DBUG_RETURN(0); > + > +retry: > + while (!(element= (TDC_element*) lf_hash_search_using_hash_value(&tdc_hash, Was lf_hash_search_using_hash_value added in a separate patch? > + thd->tdc_hash_pins, hash_value, (uchar*) key, key_length))) > { > - TABLE_SHARE *new_share; > - mysql_rwlock_unlock(&LOCK_tdc); > + TDC_element tmp(key, key_length); > + int res= lf_hash_insert(&tdc_hash, thd->tdc_hash_pins, (uchar*) &tmp); > > - if (!(new_share= alloc_table_share(db, table_name, key, key_length))) > + if (res == -1) > DBUG_RETURN(0); > - new_share->error= OPEN_FRM_OPEN_ERROR; > + else if (res == 1) > + continue; > > - mysql_rwlock_wrlock(&LOCK_tdc); > - share= (TABLE_SHARE*) my_hash_search_using_hash_value(&tdc_hash, hash_value, > - (uchar*) key, > - key_length); > - if (!share) > - { > - bool need_purge; > + element= (TDC_element*) lf_hash_search_using_hash_value(&tdc_hash, > + thd->tdc_hash_pins, hash_value, (uchar*) key, key_length); > + lf_hash_search_unpin(thd->tdc_hash_pins); How can you be sure that nobody frees your element after you unpin it? > + DBUG_ASSERT(element); > + element->dbug_release_assertions(); > > - share= new_share; > - mysql_mutex_lock(&share->tdc.LOCK_table_share); > - if (my_hash_insert(&tdc_hash, (uchar*) share)) > + if (!(share= alloc_table_share(db, table_name, key, key_length))) > { > - mysql_mutex_unlock(&share->tdc.LOCK_table_share); > - mysql_rwlock_unlock(&LOCK_tdc); > - free_table_share(share); > + lf_hash_delete(&tdc_hash, thd->tdc_hash_pins, key, key_length); > DBUG_RETURN(0); > } > - need_purge= tdc_hash.records > tdc_size; > - mysql_rwlock_unlock(&LOCK_tdc); > > /* note that tdc_acquire_share() *always* uses discovery */ > open_table_def(thd, share, flags | GTS_USE_DISCOVERY); > - share->tdc.ref_count++; > - mysql_mutex_unlock(&share->tdc.LOCK_table_share); > > if (share->error) > { > - tdc_delete_share_from_hash(share); > + free_table_share(share); > + lf_hash_delete(&tdc_hash, thd->tdc_hash_pins, key, key_length); > DBUG_RETURN(0); > } > - else if (need_purge) > + > + mysql_mutex_lock(&element->LOCK_table_share); > + element->share= share; > + share->tdc= element; > + element->ref_count++; > + element->version= tdc_refresh_version(); > + element->flushed= false; > + mysql_mutex_unlock(&element->LOCK_table_share); > + > tdc_purge(false); > if (out_table) > *out_table= 0; > share->m_psi= PSI_CALL_get_table_share(false, share); > goto end; > - } > - free_table_share(new_share); > } > > /* cannot force discovery of a cached share */ > @@ -763,18 +677,25 @@ TABLE_SHARE *tdc_acquire_share(THD *thd, const char *db, const char *table_name, > > if (out_table && (flags & GTS_TABLE)) > { > - if ((*out_table= tc_acquire_table(thd, share))) > + if ((*out_table= element->acquire_table(thd))) > { > - mysql_rwlock_unlock(&LOCK_tdc); > + lf_hash_search_unpin(thd->tdc_hash_pins); what this lf_hash_search_unpin corresponds to? > DBUG_ASSERT(!(flags & GTS_NOLOCK)); > - DBUG_ASSERT(!share->error); > - DBUG_ASSERT(!share->is_view); > - DBUG_RETURN(share); > + DBUG_ASSERT(element->share); > + DBUG_ASSERT(!element->share->error); > + DBUG_ASSERT(!element->share->is_view); > + DBUG_RETURN(element->share); > } > } > > - mysql_mutex_lock(&share->tdc.LOCK_table_share); > - mysql_rwlock_unlock(&LOCK_tdc); > + mysql_mutex_lock(&element->LOCK_table_share); > + if (!(share= element->share)) > + { > + mysql_mutex_unlock(&element->LOCK_table_share); > + lf_hash_search_unpin(thd->tdc_hash_pins); and here > + goto retry; > + } > + lf_hash_search_unpin(thd->tdc_hash_pins); may be that first lf_hash_search_unpin was wrong? the one that goes directly after lf_hash_search. > > /* > We found an existing table definition. Return it if we didn't get > @@ -983,13 +864,36 @@ bool tdc_remove_table(THD *thd, enum_tdc_remove_table_type remove_type, > thd->mdl_context.is_lock_owner(MDL_key::TABLE, db, table_name, > MDL_EXCLUSIVE)); > > - if ((share= tdc_delete_share(db, table_name))) > + > + mysql_mutex_lock(&LOCK_unused_shares); > + if (!(element= tdc_lock_share(thd, db, table_name))) > { > - I_P_List <TABLE, TABLE_share> purge_tables; > - uint my_refs= 1; > + mysql_mutex_unlock(&LOCK_unused_shares); > + DBUG_ASSERT(remove_type != TDC_RT_REMOVE_NOT_OWN_KEEP_SHARE); > + DBUG_RETURN(false); > + } > > - mysql_mutex_lock(&share->tdc.LOCK_table_share); > - tc_wait_for_mdl_deadlock_detector(share); > + DBUG_ASSERT(element != MY_ERRPTR); // What can we do about it? allocate thd->tdc_hash_pins in THD::init > + > + if (!element->ref_count) > + { > + if (element->prev) > + { > + unused_shares.remove(element); > + /* Concurrent thread may start using share again, reset prev and next. */ What concurrent thread? element->ref_count is 0, you removed the the share from the unused_shares list. There can be no concurrent threads, as far as I understand. And indeed, you do remove the share from the hash (which does free_table_share) immediately below, which also proves that there can be no concurrent threads. > + element->prev= 0; > + element->next= 0; > + } > + mysql_mutex_unlock(&LOCK_unused_shares); > + > + tdc_delete_share_from_hash(element); > + DBUG_RETURN(true); > + } > + mysql_mutex_unlock(&LOCK_unused_shares); > + > + element->ref_count++; > + > + element->wait_for_mdl_deadlock_detector(); > /* > Mark share flushed in order to ensure that it gets > automatically deleted once it is no longer referenced. Regards, Sergei

2 4

[Maria-developers] ccache with mariadb?
by Otto Kekäläinen 28 Dec '14

28 Dec '14

Hello! Has somebody managed to configure ccache correctly so that it would work with MariaDB compilation? https://ccache.samba.org/ I don't seem to get it working as the build always seems to pick up the real gcc binary instead of any of the ccache symlink wrappers I've manually put in place to replace all of gcc, cc1, cc1plus etc.

2 3

Re: [Maria-developers] MDEV-7273 - 10.1 fails to start up during tc_log initializations on PPC64
by Sergei Golubchik 26 Dec '14

26 Dec '14

Hi, Sergey! On Dec 26, svoj(a)mariadb.org wrote: > revision-id: f65901eef28728212a350f61569dc884c747bffc > parent(s): 8c616cd347aa7b3fcc2d8defb5e172ad316f432c > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2014-12-26 23:38:45 +0400 > message: > > MDEV-7273 - 10.1 fails to start up during tc_log initializations on PPC64 > > log-tc-size is 24K by default. Page size is 64K on PPC64. But log-tc-size > must be at least 3 x page size. This is enforced by TC_LOG_MMAP::open() > with a comment: to guarantee non-empty pool. > > This all makes server not startable in default configuration on PPC64. > > Autosize log-tc-size, so that it's min value= page size * 3, default > value= page size * 6, block size= page size. > > diff --git a/sql/log.cc b/sql/log.cc > index 66e1426..52c85d6 100644 > --- a/sql/log.cc > +++ b/sql/log.cc > @@ -8780,6 +8779,7 @@ int TC_LOG_MMAP::sync() > int TC_LOG_MMAP::unlog(ulong cookie, my_xid xid) > { > pending_cookies *full_buffer= NULL; > + uint32 ncookies= tc_log_page_size / sizeof(my_xid); > DBUG_ASSERT(*(my_xid *)(data+cookie) == xid); > > /* > @@ -8793,7 +8793,7 @@ int TC_LOG_MMAP::unlog(ulong cookie, my_xid xid) > mysql_mutex_lock(&LOCK_pending_checkpoint); > if (pending_checkpoint == NULL) > { > - uint32 size= sizeof(*pending_checkpoint); > + uint32 size= sizeof(*pending_checkpoint) + sizeof(ulong) * (ncookies - 1); > if (!(pending_checkpoint= > (pending_cookies *)my_malloc(size, MYF(MY_ZEROFILL)))) > { > @@ -8804,8 +8804,7 @@ int TC_LOG_MMAP::unlog(ulong cookie, my_xid xid) > } > > pending_checkpoint->cookies[pending_checkpoint->count++]= cookie; > - if (pending_checkpoint->count == sizeof(pending_checkpoint->cookies) / > - sizeof(pending_checkpoint->cookies[0])) > + if (pending_checkpoint->count == ncookies) > { > full_buffer= pending_checkpoint; > pending_checkpoint= NULL; > @@ -8839,7 +8838,7 @@ int TC_LOG_MMAP::unlog(ulong cookie, my_xid xid) > if (count == 0) > { > uint i; > - for (i= 0; i < sizeof(pending->cookies)/sizeof(pending->cookies[0]); ++i) > + for (i= 0; i < tc_log_page_size / sizeof(my_xid); ++i) I wonder why you didn't use ncookies here :) > delete_entry(pending->cookies[i]); > my_free(pending); > } Regards, Sergei

2 1

Re: [Maria-developers] MDEV-7273 - 10.1 fails to start up during tc_log initializations on PPC64
by Sergei Golubchik 26 Dec '14

26 Dec '14

Hi, Sergey! Yes, that's what I had in mind, thanks! There are few rough edges, see below. Ok to push after you fix that, there's no need for another review round. On Dec 26, svoj(a)mariadb.org wrote: > revision-id: 72e58dc2b1a0bb859e918de0ff820d5a2a92dc32 > parent(s): db89dd3a8f7b0d868946d25ba98c6f88612d309a > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2014-12-26 17:03:45 +0400 > message: > > MDEV-7273 - 10.1 fails to start up during tc_log initializations on PPC64 > > log-tc-size is 24K by default. Page size is 64K on PPC64. But log-tc-size > must be at least 3 x page size. This is enforced by TC_LOG_MMAP::open() > with a comment: to guarantee non-empty pool. > > This all makes server not startable in default configuration on PPC64. > > Autosize log-tc-size, so that it's min value= page size * 6, default > value= page size * 6, block size= page size. > > --- > mysql-test/r/mysqld--help.result | 1 - > mysql-test/suite/sys_vars/r/log_tc_size_basic.result | 4 ++++ > .../suite/sys_vars/r/sysvars_server_notembedded.result | 14 ++++++++++++++ > mysql-test/suite/sys_vars/t/log_tc_size_basic.test | 5 +++++ > mysql-test/t/mysqld--help.test | 2 +- > sql/log.cc | 5 ++--- > sql/log.h | 3 +-- > sql/mysqld.cc | 7 +------ > sql/sys_vars.cc | 11 +++++++++++ > 9 files changed, 39 insertions(+), 13 deletions(-) > > diff --git a/mysql-test/suite/sys_vars/t/log_tc_size_basic.test b/mysql-test/suite/sys_vars/t/log_tc_size_basic.test > new file mode 100644 > index 0000000..1fce212 > --- /dev/null > +++ b/mysql-test/suite/sys_vars/t/log_tc_size_basic.test > @@ -0,0 +1,5 @@ > +--error ER_INCORRECT_GLOBAL_LOCAL_VAR,ER_UNKNOWN_SYSTEM_VARIABLE > +SET GLOBAL log_tc_size=1; > + > +--error ER_INCORRECT_GLOBAL_LOCAL_VAR,ER_UNKNOWN_SYSTEM_VARIABLE do we ever build on platforms without mmap? if the answer is "no" - you don't need ER_UNKNOWN_SYSTEM_VARIABLE here > +SET SESSION log_tc_size=1; > diff --git a/mysql-test/suite/sys_vars/r/sysvars_server_notembedded.result b/mysql-test/suite/sys_vars/r/sysvars_server_notembedded.result > index 006b9a9..5491169 100644 > --- a/mysql-test/suite/sys_vars/r/sysvars_server_notembedded.result > +++ b/mysql-test/suite/sys_vars/r/sysvars_server_notembedded.result > @@ -1757,6 +1757,20 @@ NUMERIC_BLOCK_SIZE NULL > ENUM_VALUE_LIST innodb,query_plan,explain > READ_ONLY NO > COMMAND_LINE_ARGUMENT REQUIRED > +VARIABLE_NAME LOG_TC_SIZE > +SESSION_VALUE NULL > +GLOBAL_VALUE 24576 > +GLOBAL_VALUE_ORIGIN AUTO > +DEFAULT_VALUE 24576 > +VARIABLE_SCOPE GLOBAL > +VARIABLE_TYPE BIGINT UNSIGNED > +VARIABLE_COMMENT Size of transaction coordinator log. > +NUMERIC_MIN_VALUE 24576 > +NUMERIC_MAX_VALUE 18446744073709551615 > +NUMERIC_BLOCK_SIZE 4096 > +ENUM_VALUE_LIST NULL > +READ_ONLY YES > +COMMAND_LINE_ARGUMENT REQUIRED if the answer is "yes" - this test will fail there. even if the answer is "no", the variable value is architecture dependent. But it's good to see all stable variable metadata (e.g. that GLOBAL_VALUE_ORIGIN is AUTO). > VARIABLE_NAME LOG_WARNINGS > SESSION_VALUE 1 > GLOBAL_VALUE 1 > diff --git a/sql/log.cc b/sql/log.cc > index 66e1426..6d6512e 100644 > --- a/sql/log.cc > +++ b/sql/log.cc > @@ -8793,7 +8792,7 @@ int TC_LOG_MMAP::unlog(ulong cookie, my_xid xid) > mysql_mutex_lock(&LOCK_pending_checkpoint); > if (pending_checkpoint == NULL) > { > - uint32 size= sizeof(*pending_checkpoint); > + uint32 size= sizeof(*pending_checkpoint) + tc_log_page_size / sizeof(my_xid); that's not enough. There are further sizeof's, like if (pending_checkpoint->count == sizeof(pending_checkpoint->cookies) / sizeof(pending_checkpoint->cookies[0])) and for (i= 0; i < sizeof(pending->cookies)/sizeof(pending->cookies[0]); ++i) I've found these two by looking for "sizeof" in the Kristian's patch that introduced pending_checkpoint. There can be more sizeofs added later. > if (!(pending_checkpoint= > (pending_cookies *)my_malloc(size, MYF(MY_ZEROFILL)))) > { > diff --git a/sql/sys_vars.cc b/sql/sys_vars.cc > index fb11377..bfaf7d7 100644 > --- a/sql/sys_vars.cc > +++ b/sql/sys_vars.cc > @@ -5202,3 +5202,14 @@ static Sys_var_mybool Sys_strict_password_validation( > "that cannot be validated (passwords specified as a hash)", > GLOBAL_VAR(strict_password_validation), > CMD_LINE(OPT_ARG), DEFAULT(TRUE), NO_MUTEX_GUARD, NOT_IN_BINLOG); > + > +#ifdef HAVE_MMAP > +static Sys_var_ulong Sys_log_tc_size( > + "log_tc_size", > + "Size of transaction coordinator log.", > + READ_ONLY GLOBAL_VAR(opt_tc_log_size), > + CMD_LINE(REQUIRED_ARG), > + VALID_RANGE(my_getpagesize() * 6, ULONG_MAX), > + DEFAULT(my_getpagesize() * 6), > + BLOCK_SIZE(my_getpagesize())); I think, min value could be my_getpagesize()*3 ? > +#endif Regards, Sergei

2 1

Re: [Maria-developers] MDEV-7273 - 10.1 fails to start up during tc_log initializations on PPC64
by Sergei Golubchik 25 Dec '14

25 Dec '14

Hi, Sergey! On Dec 17, svoj(a)mariadb.org wrote: > revision-id: 4a469b0ed68bb341e1a1e1ba540a45a3b6ff3ef9 > parent(s): ea8862aa9ffa1cfac9a99bbe0fe9b539f12b9a19 > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2014-12-17 13:55:01 +0400 > message: > > MDEV-7273 - 10.1 fails to start up during tc_log initializations on PPC64 > > log-tc-size is 24K by default. Page size is 64K on PPC64. But log-tc-size > must be at least 3 x page size. This is enforced by TC_LOG_MMAP::open() > with a comment: to guarantee non-empty pool. > > This all makes server not startable in default configuration on PPC64. > > When creating tc log, adjust it's size to max(log-tc-size, page size * 6). > > --- > sql/log.cc | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/sql/log.cc b/sql/log.cc > index 24e5283..a2fd70f 100644 > --- a/sql/log.cc > +++ b/sql/log.cc > @@ -8316,7 +8316,7 @@ int TC_LOG_MMAP::open(const char *opt_name) > O_RDWR, MYF(MY_WME))) < 0) > goto err; > inited=1; > - file_length= opt_tc_log_size; > + file_length= MY_MAX(tc_log_page_size * 2 * 3, opt_tc_log_size); > if (mysql_file_chsize(fd, file_length, 0, MYF(MY_WME))) > goto err; > } I don't particularly like silent adjustments of user-specified values. And anyway, as you found out, this fix is not sufficient. I'd suggest the following: 1. Create a sysvar for log-tc-size 2. invoke my_getpagesize() not from log.cc, but from mysqld.cc, and set min and default values for log-tc-size appropriately, don't forget to use SYSVAR_AUTOSIZE() 3. change the code not to depend on the TC_LOG_PAGE_SIZE constant. - it's basically used in only two places now: to set a default for TC_LOG_MIN_SIZE (but you'll fix it step 2) and in kristian's code for group commit, cookies[] array. - don't mind TC_LOG_PAGE_SIZE being used in TC_LOG_BINLOG, it means nothing there, any reasonable number will do, I suppose. - so basically you only need to remove TC_LOG_PAGE_SIZE from cookies[] array. That is, you can define it as 'ulong cookies[1]' and take care to fix all sizeof() in the code to add the real size there. 4. Then you can remove that assert. The tc log page will be of my_getpagesize() size and log-tc-size will be N*my_getpagesize() where N>3. Regards, Sergei

1 0

[Maria-developers] JIRA ping
by Jonas Oreland 23 Dec '14

23 Dec '14

Hi Kristian, I light of recent "I did see any of your comments until by accident just now", I now mail you directly that I updated both MDEV-162 and MDEV-7257. I want (review) comments. /Jonas

2 9

[Maria-developers] Review of Debian packaging file listing?
by Otto Kekäläinen 22 Dec '14

22 Dec '14

Hello! I recently updated my build scripts to automatically generate a file listing for each commit in in the Debian packaging "pool" I have. Here is a recent one: http://labs.seravo.fi/~otto/mariadb-repo/mariadb-10.0-sid-amd64/filelist-ec… It is now very easy to review and track what files are in what package. Could somebody please quickly review that file and tell me if you spot: - any files in wrong package and should be moved? - any package that has too few files and maybe something missing? - any file attributes that might be wrong or insecure? - should any of the MariaDB plugins deserve to be shipped as a separate package? (apart from current connect and oqgraph) -- Check out our blog at http://seravo.fi/blog and follow @ottokekalainen

4 5

[Maria-developers] Report from Debian/Ubuntu MySQL packaging sprint in London in December
by Otto Kekäläinen 21 Dec '14

21 Dec '14

Hello! I thought I'd brief the readers of this mailing list about the packaging sprint in London last week. It was organized by Canonical and in addition to Canonical staff and me there where also people from Oracle and Percona. We mostly worked on syncing the MySQL/MariaDB/Percona server packaging and a little bit also on creating a unified Galera package. We made good progress and all results have been pushed to public repos. I also recently updated the pages https://wiki.debian.org/Teams/MySQL/MariaDB and https://wiki.debian.org/Teams/MySQL/Galera to reflect the current status. One big technical change that was discussed and planned is the decoupling of the config and data directories. The plan is that in the future no MySQL variant in particular would anymore own the /etc/mysql/my.cnf file but instead each variant would register their own config file at installation time by using the Debian update-alternatives mechanism. So in future there would be /etc/mysql/mariadb.conf, mysql.conf, percona.conf etc and some more snippets in /etc/mysql/mariadb.conf.d/.. etc. The plan is not complete nor tested, but there are preliminary changes in the mysql-common package to provide the config file registration hook and in a preliminary Percona Debian package to use the new config file registration facility. As the config file defines the data directory, successful decoupling of config files would make it easy to also decouple the data directories so that if the user installs/removes/upgrades any of the MySQL variants they would not mess up their data accidentally and the installation scripts would help users to dump/migrate/copy/move their data in the common scenarios. -- Check out our blog at http://seravo.fi/blog and follow @ottokekalainen

2 1

[Maria-developers] CREATE USER gotcha
by Alexander Barkov 19 Dec '14

19 Dec '14

Hi Ian, I think there is a CREATE USER related gotcha which is not described in the manual. CREATE USER allows to create multiple users at the same time: CREATE USER user1, user2; A reader of https://mariadb.com/kb/en/mariadb/documentation/sql-commands/account-manage… might think that the above query is a short synonym for: CREATE USER user1; CREATE USER user2; But in fact it is not always true. The difference can happen during replication. A "CREATE USER" statement is binlogged as a single shot, but only if at least one user was really created. Suppose (on master) user1 already exists, while user2 does not. "CREATE USER user1, user2" will be binary logged as is, while two independent "CREATE USER user1" and "CREARE USER user2" statements will be logged only as "CREATE USER user2". The CREATE statement for user1 won't be logged, because it failed on master! Now, suppose non of the users exist on slave. In case of "CREATE USER user1, user2;" both will be created on slave. In case of two independent queries, only "user2" will be created on slave.

4 6

[Maria-developers] Syntax for parallel replication
by Kristian Nielsen 18 Dec '14

18 Dec '14

Hi Serg, Can you help me with suggestions/comments for the following proposal for how to do configuration for MDEV-6676, speculative parallel replication? I am not too happy about how things are in 10.0. There, there is a single option --slave-parallel-threads=N. If N>0, then parallel replication is enabled, else not. One problem is that this makes it not configurable per multi-source master connection. Another problem is that there are two possible mechanisms for parallelisation, group-commit based and domain_id based; currently, one can enable none of them or both, but not just one of them. MDEV-6676 will introduce at least one other mechanism, which seems to make it essential to make a better way to configure this. Now, ideally, there would not be any configuration at all. The server would just run things in parallel when possible and to the degree desirable. However, I think there are some reasons that we need to allow the user this configurability: - Parallel replication is still a somewhat experimental feature, so it seems too risky to enable it by default. Also, it doesn't really seem possible for the server to automatically set the best number of threads to use, with current implementation (or possibly any implementation). - When replicating with non-transactional updates, or in non-gtid mode, slave state is not crash safe. This is true in non-parallel replication also, but in parallel replication, the problem seems amplified, as there may be multiple transactions in progress at the time of a crash, complicating possible manual recovery. This also suggests that parallel replication must be configurable. - When using domain-based parallel replication, the user is responsible for ensuring that independent domains are non-conflicting and can be replicated out-of-order wrt. each other. So if replication domains are used, but this property is not guaranteed, then domain-based parallel replication need to be configurable, or parallel replication cannot be used at all. - The new speculative replication feature in MDEV-6676 is not always guaranteed to be a win - in some workloads, where there are many conflicts between successive transactions, excessive rollback could cause it to be less efficient than not using it. Again, this suggests it needs to be configurable. So given this, I came up with the following idea for syntax: CHANGE MASTER TO PARALLEL_MODE=(domain,groupcommit,transactional,waiting) Each of the four keywords in the parenthesis is optional. "domain" enables domain-based parallelisation, where each replication domain is treated independently. "groupcommit" enables the non-speculative mode, where only transactions that group-committed together on the master are applied in parallel on the slave. "transactional" enables the speculative mode, where all transactional DML is optimistically tried in parallel, and then in case of conflict a rollback and retry is done. "groupcommit" and "transactional" are mutually exclusive, at most one of them can be specified. The default would be (domain,groupcommit) to be back-wards compatible with 10.0. If slave_parallel_thread=0, then no parallel apply will happen even if PARALLEL_MODE is non-empty. If slave_parallel_thread>0 but PARALLEL_MODE is empty (PARALLEL_MODE=()), then again no parallel apply will be done. The "waiting" option is not essential to add, we could remove it, I put it in because there were already a number of options so it seemed to cause no harm. The idea is that on the master, we detect if transaction T2 had to do a row lock wait on transaction T1. If so, it seems likely that a similar conflict could occur on the slave, so we will not run T2 in parallel with T1; instead we will let T2 wait for T1 to commit before T2 is started. The "waiting" option could be enabled by a user to disable this check, enabling even more aggressive parallelisation. I am not sure if it is worth it to have this configurable though, comments welcome. I have not checked how hard it will be to implement the new syntax. We do not have any similar multi-option CHANGE MASTER elements, as far as I know, but it is similar to ENUM system variables. And we already have a IGNORE_SERVER_IDS syntax with comma-separated list within parenthesis. So hopefully not too hard to do, and somewhat consistent with existing syntax. So what do you think? Is it a reasonable syntax? Any comments, or suggestions for better way to do it? Thanks, - Kristian.

5 23

Re: [Maria-developers] Rev 4383: MDEV-6916:Upgrade from MySQL to MariaDB breaks already created views
by Sergei Golubchik 17 Dec '14

17 Dec '14

Hi, Sanja! Comments about the code, see below. Comments about the whole approach - see my other email. On Dec 07, sanja(a)askmonty.org wrote: > At file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6916/ > > ------------------------------------------------------------ > revno: 4383 > revision-id: sanja(a)askmonty.org-20141207094825-tx163unk3mm4gkql > parent: sanja(a)askmonty.org-20141203234414-63zkpeq9eb0n9lc1 > committer: sanja(a)askmonty.org > branch nick: work-maria-5.5-MDEV-6916 > timestamp: Sun 2014-12-07 10:48:25 +0100 > message: > MDEV-6916:Upgrade from MySQL to MariaDB breaks already created views > (SQL-level commands to support view update) > === modified file 'mysql-test/r/view.result' > --- a/mysql-test/r/view.result 2014-12-03 22:37:59 +0000 > +++ b/mysql-test/r/view.result 2014-12-07 09:48:25 +0000 > @@ -5398,6 +5398,29 @@ DROP VIEW v1; > DROP TABLE t1, t2; > create view v1 as select 1; > drop view v1; > +# > +#MDEV-6916:Upgrade from MySQL to MariaDB breaks already created views > +# (SQL command for upgrade check) > +# > +create table t1 (a int); > +create algorithm=temptable view v1 as select a from t1; > +show create view v1; > +View Create View character_set_client collation_connection > +v1 CREATE ALGORITHM=TEMPTABLE DEFINER=`root`@`localhost` SQL SECURITY DEFINER VIEW `v1` AS select `t1`.`a` AS `a` from `t1` latin1 latin1_swedish_ci > +check table v1 force mariadb upgrade; > +Table Op Msg_type Msg_text > +test.v1 check note View algorithm swap > +show create view v1; > +View Create View character_set_client collation_connection > +v1 CREATE ALGORITHM=MERGE DEFINER=`root`@`localhost` SQL SECURITY DEFINER VIEW `v1` AS select `t1`.`a` AS `a` from `t1` latin1 latin1_swedish_ci No, I don't like that. FORCE or not, upgrading of a view should not blindly change the alorithm. This functionality is created to solve a specific problem. So when a view has mariadb-version field we know with certainty that the algorithm is correct and it should *not* be changed with or without FORCE. In the test you'll need to copy mysql-created view frm (from std_data). Addendum: ok, I've read till the end. See my comment in view_check() function. > +check table v1 force mariadb upgrade; Also, I don't like that CHECK changes the algorithm of the view. For tables CHECK only performs the *check* (and updates the version in the frm). It's REPAIR that does the change. To be consistent with the behavior for tables, CHECK ... FOR UPGRADE should only report an error, like "view needs upgrading" and a user should then use REPAIR to actually fix the view. > +Table Op Msg_type Msg_text > +test.v1 check note View algorithm swap > +show create view v1; > +View Create View character_set_client collation_connection > +v1 CREATE ALGORITHM=TEMPTABLE DEFINER=`root`@`localhost` SQL SECURITY DEFINER VIEW `v1` AS select `t1`.`a` AS `a` from `t1` latin1 latin1_swedish_ci > +drop view v1; > +drop table t1; > # ----------------------------------------------------------------- > # -- End of 5.5 tests. > # ----------------------------------------------------------------- > > === modified file 'sql/share/errmsg-utf8.txt' > --- a/sql/share/errmsg-utf8.txt 2014-05-27 06:45:01 +0000 > +++ b/sql/share/errmsg-utf8.txt 2014-12-07 09:48:25 +0000 > @@ -6565,3 +6565,5 @@ ER_QUERY_EXCEEDED_ROWS_EXAMINED_LIMIT > ER_NO_SUCH_TABLE_IN_ENGINE 42S02 > eng "Table '%-.192s.%-.192s' doesn't exist in engine" > swe "Det finns ingen tabell som heter '%-.192s.%-.192s' i handlern" > +ER_VIEW_UPGRADE_IS_DONE > + eng "View algorithm swap" No way, you cannot add a new error message in 5.5 because the next major branch (10.0) is already GA. > === modified file 'sql/sql_admin.cc' > --- a/sql/sql_admin.cc 2014-05-03 16:12:17 +0000 > +++ b/sql/sql_admin.cc 2014-12-07 09:48:25 +0000 > @@ -314,7 +314,8 @@ static bool mysql_admin_table(THD* thd, > HA_CHECK_OPT *), > int (handler::*operator_func)(THD *, > HA_CHECK_OPT *), > - int (view_operator_func)(THD *, TABLE_LIST*)) > + int (view_operator_func)(THD *, TABLE_LIST*, > + HA_CHECK_OPT *)) > { > TABLE_LIST *table; > SELECT_LEX *select= &thd->lex->select_lex; > @@ -521,7 +522,7 @@ static bool mysql_admin_table(THD* thd, > ER_CHECK_NO_SUCH_TABLE, ER(ER_CHECK_NO_SUCH_TABLE)); > /* if it was a view will check md5 sum */ > if (table->view && > - view_checksum(thd, table) == HA_ADMIN_WRONG_CHECKSUM) > + view_check(thd, table, check_opt) == HA_ADMIN_WRONG_CHECKSUM) > push_warning(thd, MYSQL_ERROR::WARN_LEVEL_WARN, > ER_VIEW_CHECKSUM, ER(ER_VIEW_CHECKSUM)); > if (thd->stmt_da->is_error() && > @@ -536,7 +537,7 @@ static bool mysql_admin_table(THD* thd, > if (table->view) > { > DBUG_PRINT("admin", ("calling view_operator_func")); > - result_code= (*view_operator_func)(thd, table); > + result_code= (*view_operator_func)(thd, table, check_opt); > goto send_result; > } > > @@ -849,6 +850,14 @@ send_result_message: > system_charset_info); > break; > } > + case HA_ADMIN_VIEW_UPGRADE_IS_DONE: > + { > + protocol->store(STRING_WITH_LEN("note"), system_charset_info); > + protocol->store(ER(ER_VIEW_UPGRADE_IS_DONE), > + strlen(ER(ER_VIEW_UPGRADE_IS_DONE)), > + system_charset_info); > + break; > + } Don't do that. First, you cannot have ER_VIEW_UPGRADE_IS_DONE. Second, it's simply unnecessary. Return HA_ADMIN_OK from the view_operator_func() and use push_warning inside view_operator_func() to say that the algorithm was changed. > case HA_ADMIN_NEEDS_UPGRADE: > case HA_ADMIN_NEEDS_ALTER: > @@ -1071,7 +1080,7 @@ bool Check_table_statement::execute(THD > > res= mysql_admin_table(thd, first_table, &m_lex->check_opt, "check", > lock_type, 0, 0, HA_OPEN_FOR_REPAIR, 0, > - &handler::ha_check, &view_checksum); > + &handler::ha_check, &view_check); > > m_lex->select_lex.table_list.first= first_table; > m_lex->query_tables= first_table; > > === modified file 'sql/sql_view.cc' > --- a/sql/sql_view.cc 2014-12-03 23:44:14 +0000 > +++ b/sql/sql_view.cc 2014-12-07 09:48:25 +0000 > @@ -729,9 +729,29 @@ err: > DBUG_RETURN(res || thd->is_error()); > } > > +static void make_view_filename(LEX_STRING *dir, char *dir_buff, > + size_t dir_buff_len, > + LEX_STRING *path, char *path_buff, > + size_t path_buff_len, > + LEX_STRING *file, > + TABLE_LIST *view) > +{ > + /* print file name */ > + dir->length= build_table_filename(dir_buff, dir_buff_len - 1, > + view->db, "", "", 0); > + dir->str= dir_buff; > + > + path->length= build_table_filename(path_buff, path_buff_len - 1, > + view->db, view->table_name, reg_ext, 0); > + path->str= path_buff; > + > + file->str= path->str + dir->length; > + file->length= path->length - dir->length; > +} > + > > /* number of required parameters for making view */ > -static const int required_view_parameters= 15; > +static const int required_view_parameters= 16; eh? why? > /* > table of VIEW .frm field descriptors > @@ -795,6 +815,65 @@ static File_option view_parameters[]= > static LEX_STRING view_file_type[]= {{(char*) STRING_WITH_LEN("VIEW") }}; > > > +int mariadb_fix_view(THD *thd, TABLE_LIST *view) > +{ > + char dir_buff[FN_REFLEN + 1], path_buff[FN_REFLEN + 1]; > + LEX_STRING dir, file, path; > + DBUG_ENTER("mariadb_fix_view"); > + > + if (view->algorithm == VIEW_ALGORITHM_UNDEFINED) > + DBUG_RETURN(HA_ADMIN_OK); > + > + make_view_filename(&dir, dir_buff, sizeof(dir_buff), > + &path, path_buff, sizeof(path_buff), > + &file, view); > + /* init timestamp */ > + if (!view->timestamp.str) > + view->timestamp.str= view->timestamp_buffer; > + > + /* check old .frm */ > + { > + char path_buff[FN_REFLEN]; > + LEX_STRING path; > + File_parser *parser; > + > + path.str= path_buff; > + fn_format(path_buff, file.str, dir.str, "", MY_UNPACK_FILENAME); > + path.length= strlen(path_buff); > + if (access(path.str, F_OK)) > + DBUG_RETURN(HA_ADMIN_INVALID); > + > + if (!(parser= sql_parse_prepare(&path, thd->mem_root, 0))) > + DBUG_RETURN(HA_ADMIN_INTERNAL_ERROR); > + > + if (!parser->ok() || !is_equal(&view_type, parser->type())) > + DBUG_RETURN(HA_ADMIN_INVALID); > + } > + > + DBUG_ASSERT(view->algorithm == VIEW_ALGORITHM_MERGE || > + view->algorithm == VIEW_ALGORITHM_TMPTABLE); if() instead of an assert, please. while unlikely, it's possible that the file would be changed between two opens. > + if (view->algorithm == VIEW_ALGORITHM_MERGE) > + view->algorithm= VIEW_ALGORITHM_TMPTABLE; > + else > + view->algorithm= VIEW_ALGORITHM_MERGE; > + > + if (sql_create_definition_file(&dir, &file, view_file_type, > + (uchar*)view, view_parameters)) > + { > + sql_print_error("View '%-.192s'.'%-.192s': algorithm swap error.", "View %`s.%`s: cannot change the algorithm from %s to %s" And push_warning_printf() with the same. Or, rather, only push_warning_printf without sql_print_information. > + view->db, view->table_name); > + DBUG_RETURN(HA_ADMIN_INTERNAL_ERROR); > + } > + sql_print_information("View '%-.192s'.'%-.192s': algorithm swapped to '%s'", "View %`s.%`s: algorithm changed from %s to %s" And push_warning_printf() with the same. Or, rather, only push_warning_printf without sql_print_information. > + view->db, view->table_name, > + (view->algorithm == VIEW_ALGORITHM_MERGE)? > + "MERGE":"TEMPTABLE"); > + > + > + DBUG_RETURN(HA_ADMIN_VIEW_UPGRADE_IS_DONE); > +} > + > + > /* > Register VIEW (write .frm & process .frm's history backups) > > @@ -931,16 +1010,9 @@ static int mysql_register_view(THD *thd, > } > loop_out: > /* print file name */ > - dir.length= build_table_filename(dir_buff, sizeof(dir_buff) - 1, > - view->db, "", "", 0); > - dir.str= dir_buff; > - > - path.length= build_table_filename(path_buff, sizeof(path_buff) - 1, > - view->db, view->table_name, reg_ext, 0); > - path.str= path_buff; > - > - file.str= path.str + dir.length; > - file.length= path.length - dir.length; > + make_view_filename(&dir, dir_buff, sizeof(dir_buff), > + &path, path_buff, sizeof(path_buff), > + &file, view); > > /* init timestamp */ > if (!view->timestamp.str) > @@ -1953,6 +2025,33 @@ int view_checksum(THD *thd, TABLE_LIST * > HA_ADMIN_OK); > } you can make view_checksum() static now. > > +/** > + Check view > + > + @param thd thread handle > + @param view view for check > + @param check_opt check options > + > + @retval HA_ADMIN_OK OK > + @retval HA_ADMIN_NOT_IMPLEMENTED it is not VIEW > + @retval HA_ADMIN_WRONG_CHECKSUM check sum is wrong > +*/ > +int view_check(THD *thd, TABLE_LIST *view, HA_CHECK_OPT *check_opt) > +{ > + int res; > + DBUG_ENTER("view_check"); > + if ((res= view_checksum(thd, view)) != HA_ADMIN_OK) > + DBUG_RETURN(res); > + if (((check_opt->sql_flags & TT_MARIADB_UPGRADE) && > + !view->mariadb_version) || > + ((check_opt->sql_flags & (TT_MARIADB_UPGRADE | TT_FORCE)))) What is the point? Why would one ever want to change the algorithm of the provably created-in-mariadb view and pretend that it's an "upgrade"? To change the algorithm one should use ALTER. Fixing the view after an upgrade is not altering it, it's *fixing after an upgrade*. And views that were created in mariadb don't need to be fixed after an upgrade. No FORCE please. > + { > + DBUG_PRINT("XXX", ("Upgrade")); please either remove this DBUG_PRINT or change it to use a reasonable keyword. > + DBUG_RETURN(mariadb_fix_view(thd, view)); > + } > + DBUG_RETURN(HA_ADMIN_OK); > +} > + > === modified file 'sql/sql_yacc.yy' > --- a/sql/sql_yacc.yy 2014-11-08 18:54:42 +0000 > +++ b/sql/sql_yacc.yy 2014-12-07 09:48:25 +0000 > @@ -7299,6 +7300,10 @@ mi_check_type: > | EXTENDED_SYM { Lex->check_opt.flags|= T_EXTEND; } > | CHANGED { Lex->check_opt.flags|= T_CHECK_ONLY_CHANGED; } > | FOR_SYM UPGRADE_SYM { Lex->check_opt.sql_flags|= TT_FOR_UPGRADE; } > + | MARIADB_SYM UPGRADE_SYM > + { Lex->check_opt.sql_flags|= TT_MARIADB_UPGRADE; } > + | FORCE_SYM MARIADB_SYM UPGRADE_SYM > + { Lex->check_opt.sql_flags|= (TT_MARIADB_UPGRADE | TT_FORCE); } > ; > > optimize: Regards, Sergei

1 0

Re: [Maria-developers] Rev 4384: MDEV-6916: Upgrade from MySQL to MariaDB breaks already created views
by Sergei Golubchik 17 Dec '14

17 Dec '14

Hi, Sanja! Ok, I've seen all four patches. Specific comments about the code are below and in other emails. But the main question is - why are you doing it, what problem does it solve. As you said, the use case is - a user upgrades *from MySQL* and *runs mysql_upgrade*. To cover this use case mysql_upgrade needs to 1) detect whether it's an upgrade from MySQL, and to 2) run a command to change the algorithm of all views. You've done a lot of changes that aren't related to that. If this is, indeed, the use case, than you only need to: 1. add MySQL detection in mysql_upgrade. You have it, is_mysql() function. 2. add the code to toggle the algorithm of all views. See? No command line options, no FORCE, no CHECK, nothing. Now, let's consider extensions to this minimal functionality. 3. mariadb-version in the view frm file. this is a good idea, makes views a lot more future-proof. ok. I don't see why other extensions might be needed. Now, how to toggle the algorithm. The most natural solution would be to use ALTER VIEW. Like ALTER VIEW v1 ALGORITHM=MERGE. Unfortunately, ALTER VIEW doesn't work that way, I don't know why it was implemented in that crazy way in MySQL, but it's mostly useless. CHECK should not change its arguments. REPAIR? May be. I'd still prefer ALTER, but I don't know whether it can be extended in 5.5 in a safe way. If not then REPAIR ... AFTER UPGRADE is probably the second best. Or even REPAIR VIEW ... AFTER UPGRADE. It won't conflict with anything, doesn't add new keywords. Drawback - we'll be stuck with that hack for quite a while :( Even better would be to make mysql_upgrade to scan the data directory and rewrite view frms. No hacks or new grammar in the server. No new features in 5.5 GA. Nothing to maintain for years. Almost perfect. But it requires mysql_upgrade to have write access to the datadir. It kind of needs it already, but it's optional. With this approach it'll be less optional than before. So, I don't know... The most reasonable and future-proof solution would be to use ALTER TABLE. But I don't know whether we'll be able to do it in 5.5, looks a bit risky. Then there are reasonably safe hacks like REPAIR VIEW. And most safe and quite clean mysql_upgrade only approach, that needs write access to the datadir. Opinions? Anyway, here are comments about the code, below: On Dec 11, sanja(a)askmonty.org wrote: > At file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6916/ > > ------------------------------------------------------------ > revno: 4384 > revision-id: sanja(a)askmonty.org-20141211180458-j8ajlibnvvydmj40 > parent: sanja(a)askmonty.org-20141207094825-tx163unk3mm4gkql > committer: sanja(a)askmonty.org > branch nick: work-maria-5.5-MDEV-6916 > timestamp: Thu 2014-12-11 19:04:58 +0100 > message: > MDEV-6916: Upgrade from MySQL to MariaDB breaks already created views > > mysql_upgrade now fix views from mysql. > > (+ fix new view detection) > === modified file 'client/mysql_upgrade.c' > --- a/client/mysql_upgrade.c 2014-03-27 21:26:58 +0000 > +++ b/client/mysql_upgrade.c 2014-12-11 18:04:58 +0000 > @@ -40,7 +40,8 @@ static char mysql_path[FN_REFLEN]; > static char mysqlcheck_path[FN_REFLEN]; > > static my_bool opt_force, opt_verbose, debug_info_flag, debug_check_flag, > - opt_systables_only, opt_version_check; > + opt_systables_only, opt_version_check, > + opt_mysql_upgrade= 0, opt_skip_mysql_upgrade= 0; > static my_bool opt_not_used, opt_silent; > static uint my_end_arg= 0; > static char *opt_user= (char*)"root"; > @@ -150,6 +151,14 @@ static struct my_option my_long_options[ > &opt_not_used, &opt_not_used, 0, GET_BOOL, NO_ARG, 1, 0, 0, 0, 0, 0}, > {"version", 'V', "Output version information and exit.", 0, 0, 0, > GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0}, > + {"mysql-upgrade", 'y', > + "Skip automatic detection MySQL and assume that we upgrade it", > + &opt_mysql_upgrade, &opt_mysql_upgrade, 0, GET_BOOL, NO_ARG, > + 0, 0, 0, 0, 0, 0}, > + {"skip-mysql-upgrade", 'Y', > + "Skip view algorithm upgrade from MySQL", > + &opt_skip_mysql_upgrade, &opt_skip_mysql_upgrade, 0, GET_BOOL, NO_ARG, > + 0, 0, 0, 0, 0, 0}, Those two are very bad option names. First, $ mysql_upgrade --mysql-upgrade looks plain weird. $ mysql_upgrade --skip-mysql-upgrade is even worse. And what about $ mysql_upgrade --mysql-upgrade --skip-mysql-upgrade Second, "skip" is the prefix automatically recognized by my_getopt. Normally, --skip-xxx is the same as --xxx=0. But now you're introducing --skip-mysql-upgrade which is very different from --skip-mysql-upgrade. While simultaneously adding support for --skip-skip-mysql-upgrade. A better option would be --fix-view-algorithm = { AUTO | ALWAYS | NEVER } > {"version-check", 'k', "Run this program only if its \'server version\' " > "matches the version of the server to which it's connecting, (enabled by " > "default); use --skip-version-check to avoid this check. Note: the \'server " > @@ -754,6 +771,23 @@ static int run_mysqlcheck_upgrade(void) > NULL); > } > > +static int run_mysqlcheck_views(void) > +{ > + if (!opt_mysql_upgrade) > + return 0; > + verbose("Phase 0: Fixing views"); No "Phase 0" please, renumber all phases to start from 1. > + print_conn_args("mysqlcheck"); > + return run_tool(mysqlcheck_path, > + NULL, /* Send output from mysqlcheck directly to screen */ > + "--no-defaults", > + ds_args.str, > + "--all-databases", > + "--view-upgrade", > + opt_verbose ? "--verbose": "", > + opt_silent ? "--silent": "", > + "2>&1", > + NULL); > +} > > === modified file 'client/mysqlcheck.c' > --- a/client/mysqlcheck.c 2014-02-17 10:00:51 +0000 > +++ b/client/mysqlcheck.c 2014-12-11 18:04:58 +0000 > @@ -42,7 +42,8 @@ static my_bool opt_alldbs = 0, opt_check > opt_medium_check = 0, opt_quick = 0, opt_all_in_1 = 0, > opt_silent = 0, opt_auto_repair = 0, ignore_errors = 0, > tty_password= 0, opt_frm= 0, debug_info_flag= 0, debug_check_flag= 0, > - opt_fix_table_names= 0, opt_fix_db_names= 0, opt_upgrade= 0; > + opt_fix_table_names= 0, opt_fix_db_names= 0, opt_upgrade= 0, > + opt_mariadb_upgrade= 0, opt_f_mariadb_upgrade= 0; > static my_bool opt_write_binlog= 1, opt_flush_tables= 0; > static uint verbose = 0, opt_mysql_port=0; > static int my_end_arg; > @@ -196,6 +197,12 @@ static struct my_option my_long_options[ > NO_ARG, 0, 0, 0, 0, 0, 0}, > {"version", 'V', "Output version information and exit.", 0, 0, 0, GET_NO_ARG, > NO_ARG, 0, 0, 0, 0, 0, 0}, > + {"view-upgrade", 'f', > + "Fix view algorithm view field if it is not new MariaDB view.", "Repair algorithm value for views created by MySQL (or MariaDB before 5.5.41)" > + 0, 0, 0, GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0}, > + {"force-view-upgrade", 'F', > + "Swap view algorithm view field.", "Always change view algorithm" > + 0, 0, 0, GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0}, > {0, 0, 0, 0, 0, 0, GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0} > }; Regards, Sergei

1 0

[Maria-developers] Invisible Indexes and .frm Change?
by James Briggs 16 Dec '14

16 Dec '14

Hi list. I'm looking into adding invisible indexes. There's 3 "scopes" I've thought of: 1) global and persistent (create table, alter table, create index) 2) global temporary (alter table t1 change index invisible global) 3) session temporary (alter table t1 change index invisible session) #1 likely means changing the .frm format, which would cause backwardcompatibility issues. Are there any upcoming changes to .frm planned? Any comments or suggestions? Thanks, James Briggs. -- Cassandra/MySQL DBA. Available in San Jose area or remote. cass_top: https://github.com/jamesbriggs/cassandra-top

2 1

Re: [Maria-developers] Rev 4350: MDEV-6985: MariaDB crashes on stored procedure call
by Sergei Golubchik 16 Dec '14

16 Dec '14

Hi, Sanja! On Nov 17, sanja(a)askmonty.org wrote: > ------------------------------------------------------------ > revno: 4350 > revision-id: sanja(a)askmonty.org-20141117132635-gf281ipiomifw1oc > parent: psergey(a)askmonty.org-20141113111559-cuhsyfhor141qo6a > committer: sanja(a)askmonty.org > branch nick: work-maria-5.5-MDEV-6985 > timestamp: Mon 2014-11-17 14:26:35 +0100 > message: > MDEV-6985: MariaDB crashes on stored procedure call > > Item_ident fixed to allow double cleanup(). > === modified file 'sql/item.cc' > --- a/sql/item.cc 2014-10-06 17:53:55 +0000 > +++ b/sql/item.cc 2014-11-17 13:26:35 +0000 > @@ -862,12 +862,20 @@ void Item_ident::cleanup() > field_name ? field_name : "(null)", > orig_field_name ? orig_field_name : "(null)")); > #endif Before pushing, please remove the above #ifdef, in a separate commit. thanks! > + bool was_fixed= fixed; > Item::cleanup(); > db_name= orig_db_name; > table_name= orig_table_name; > field_name= orig_field_name; > /* Store if this Item was depended */ > - can_be_depended= test(depended_from); > + if (was_fixed) > + { > + /* > + We can trust that depended_from set correctly only if this item > + was fixed > + */ > + can_be_depended= test(depended_from); > + } Are double cleanups generally normal? Why this item is cleaned up twice? > DBUG_VOID_RETURN; > } Regards, Sergei

2 2

Re: [Maria-developers] Rev 4382: Added creation time to the view frm
by Sergei Golubchik 16 Dec '14

16 Dec '14

Hi, Sanja! On Dec 03, sanja(a)askmonty.org wrote: > At file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6916/ > > ------------------------------------------------------------ > revno: 4382 > revision-id: sanja(a)askmonty.org-20141203234414-63zkpeq9eb0n9lc1 > parent: sanja(a)askmonty.org-20141203223759-316k8y35qdb4cgyq > committer: sanja(a)askmonty.org > branch nick: work-maria-5.5-MDEV-6916 > timestamp: Thu 2014-12-04 00:44:14 +0100 > message: > Added creation time to the view frm. > ALTER VIEW saves it. CREATE VIEW & CREATE OR REPLACE VIEW reset it. Please no. This has nothing to do with MDEV-6916, it's a separate task and should be reviewed and discussed separately, not sneaked in under the MDEV-6916 cover. Regards, Sergei

1 0

Re: [Maria-developers] Rev 4381: Adding mariadb-version on the view creation to view frm. (MDEV-6916 followup)
by Sergei Golubchik 16 Dec '14

16 Dec '14

Hi, Sanja! On Dec 03, sanja(a)askmonty.org wrote: > ------------------------------------------------------------ > revno: 4381 > revision-id: sanja(a)askmonty.org-20141203223759-316k8y35qdb4cgyq > parent: jplindst(a)mariadb.org-20141203113839-xkxdy39mq77vu99y > committer: sanja(a)askmonty.org > branch nick: work-maria-5.5-MDEV-6916 > timestamp: Wed 2014-12-03 23:37:59 +0100 > message: > Adding mariadb-version on the view creation to view frm. (MDEV-6916 followup) Looks ok, thanks Regards, Sergei

1 0

[Maria-developers] Review of MDEV-7112 Split HA_CREATE_INFO
by Sergei Golubchik 04 Dec '14

04 Dec '14

Hi. It think the split is good, I liked it. I didn't like that you've tried to put the refactoring of sql_db.cc and sql_lang.h in the same commit, please move it to a separate patch. MDEV-5359 should not depend on that. Anyway, see all my comments below: > diff --git a/sql/structs.h b/sql/structs.h > index 99561c5..b38685c 100644 > --- a/sql/structs.h > +++ b/sql/structs.h > @@ -471,4 +471,67 @@ class Discrete_intervals_list { > Discrete_interval* get_current() const { return current; }; > }; > > + > +struct DDL_options_st > +{ > +public: > + enum Options > + { > + OPT_NONE= 0, > + OPT_IF_NOT_EXISTS= 2, // CREATE TABLE IF NOT EXISTS > + OPT_LIKE= 4, // CREATE TABLE LIKE > + OPT_OR_REPLACE= 16, // CREATE OR REPLACE TABLE > + OPT_OR_REPLACE_SLAVE_GENERATED= 32,// REPLACE was added on slave, it was > + // not in the original query on master. > + OPT_IF_EXISTS= 64 > + }; > + > +private: > + Options m_options; > + > +protected: > + > +public: > + Options create_like_options() const > + { > + return (DDL_options_st::Options) > + (((uint) m_options) & (OPT_IF_NOT_EXISTS | OPT_OR_REPLACE)); > + } > + void init() { m_options= OPT_NONE; } > + void init(Options options) { m_options= options; } > + void set(Options other) > + { > + m_options= other; > + } > + void set(const DDL_options_st other) > + { > + m_options= other.m_options; > + } > + bool if_not_exists() const { return m_options & OPT_IF_NOT_EXISTS; } > + bool or_replace() const { return m_options & OPT_OR_REPLACE; } > + bool or_replace_slave_generated() const > + { return m_options & OPT_OR_REPLACE_SLAVE_GENERATED; } > + bool like() const { return m_options & OPT_LIKE; } > + bool if_exists() const { return m_options & OPT_IF_EXISTS; } > + void add(const DDL_options_st::Options other) > + { > + m_options= (Options) ((uint) m_options | (uint) other); > + } > + void add(const DDL_options_st &other) > + { > + add(other.m_options); > + } > + DDL_options_st operator|(const DDL_options_st &other) > + { > + add(other.m_options); > + return *this; > + } > + DDL_options_st operator|=(DDL_options_st::Options other) > + { > + add(other); > + return *this; > + } > +}; I don't particularly like this style of C++ing... It's way too verbose. but ok > + > + > #endif /* STRUCTS_INCLUDED */ > diff --git a/sql/handler.h b/sql/handler.h > index 0044556..7b6a5f4 100644 > --- a/sql/handler.h > +++ b/sql/handler.h > @@ -376,10 +376,7 @@ enum enum_alter_inplace_result { > #define HA_KEY_BLOB_LENGTH 2 > > #define HA_LEX_CREATE_TMP_TABLE 1 > -#define HA_LEX_CREATE_IF_NOT_EXISTS 2 > -#define HA_LEX_CREATE_TABLE_LIKE 4 > #define HA_CREATE_TMP_ALTER 8 > -#define HA_LEX_CREATE_REPLACE 16 One empty line here, please, HA_MAX_REC_LENGTH is not a flag > #define HA_MAX_REC_LENGTH 65535 > > /* Table caching type */ > @@ -1571,9 +1568,9 @@ enum enum_stats_auto_recalc { HA_STATS_AUTO_RECALC_DEFAULT= 0, > HA_STATS_AUTO_RECALC_ON, > HA_STATS_AUTO_RECALC_OFF }; > Could you add structure comments here? Like what you have in MDEV, "This will be passed to mysql_create_db, mysql_drop_db, mysqld_show_create_db", etc To explain what every structure is for, what kind of data it should store, and expected users (internal DB DDL, internal table DDL, handler, etc) > -struct HA_CREATE_INFO > +struct Table_contents_source_st > { > - CHARSET_INFO *table_charset, *default_table_charset; > + CHARSET_INFO *table_charset; > LEX_CUSTRING tabledef_version; > LEX_STRING connect_string; > const char *password, *tablespace; > @@ -1635,6 +1635,55 @@ struct HA_CREATE_INFO > }; > comments for all new structures, please > +struct Schema_specification_st > +{ > + CHARSET_INFO *default_table_charset; > + void init() > + { > + bzero(this, sizeof(*this)); > + } > +}; > + > + > +struct HA_CREATE_INFO: public Table_contents_source_st, > + public Schema_specification_st > +{ > + void init() > + { > + Table_contents_source_st::init(); > + Schema_specification_st::init(); > + } > +}; > + > + > +struct Table_specification_st: public HA_CREATE_INFO, > + public DDL_options_st > +{ > + // Deep initialization > + void init() > + { > + HA_CREATE_INFO::init(); > + DDL_options_st::init(); > + } > + void init(DDL_options_st::Options options) > + { > + HA_CREATE_INFO::init(); > + DDL_options_st::init(options); > + } > + /* > + Quick initialization, for parser. > + Most of the HA_CREATE_INFO is left uninitialized. > + It gets fully initialized in sql_yacc.yy, only when the parser > + scans a related keyword (e.g. CREATE, ALTER). > + */ > + void lex_start() > + { > + HA_CREATE_INFO::options= 0; > + DDL_options_st::init(); > + } > +}; > + > + > /** > In-place alter handler context. > > diff --git a/sql/sql_lang.h b/sql/sql_lang.h > new file mode 100644 > index 0000000..76a1340 > --- /dev/null > +++ b/sql/sql_lang.h > @@ -0,0 +1,79 @@ > +/* Copyright (c) 2014, MariaDB Foundation > + > + This program is free software; you can redistribute it and/or modify > + it under the terms of the GNU General Public License as published by > + the Free Software Foundation; version 2 of the License. > + > + This program is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + GNU General Public License for more details. > + > + You should have received a copy of the GNU General Public License > + along with this program; if not, write to the Free Software > + Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */ > + > +#ifndef SQL_LANG_INCLUDED > +#define SQL_LANG_INCLUDED > + > +namespace SQL { > + > +class Identifier: protected LEX_CSTRING > +{ > + void set(const LEX_STRING &other) > + { > + str= other.str; > + length= other.length; > + DBUG_ASSERT(str[length] == 0); > + } > +public: > + Identifier(const LEX_STRING &other) > + { > + set(other); > + } > + Identifier(const LEX *); > + const char *name() const { return str; } > + const size_t name_length() const { return length; } > +}; Eh? What all this has to do with MDEV-7112 "Split HA_CREATE_INFO" ? it's not mentioned there. If it's unrelated refactoring, please, put it in a separate patch. > + > + > +class Identifier_normalized: public Identifier > +{ > + char m_buf[NAME_LEN + 1 + MYSQL50_TABLE_NAME_PREFIX_LENGTH + 10]; > + void normalize(); > +public: > + Identifier_normalized(const LEX_STRING &other) > + :Identifier(other) { normalize(); } > + Identifier_normalized(const LEX *lex) > + :Identifier(lex) { normalize(); } > +}; > + > + > +namespace Statement { > + > + > +/* > + All SQL statements that can be executed directly. > + Statement instances are created using the data in thd->lex, > + previously populated by the SQL parser. > +*/ > +class Directly_executable > +{ > +public: > + /* > + Perform all stages of a direct SQL execution: > + - Check for validity (e.g. name in "CREATE SCHEMA name" is valid). > + - Check access > + - Apply filters (e.g. Rpl_filters on slave) > + - Perform WSREP_TO_ISOLATION > + - Execute the actual action > + */ > + virtual int exec_direct(THD *thd) const = 0; > +}; > + > +} // End of namescpace Statement > + > + > +} // End of namespace SQL > + > +#endif /* SQL_LANG_INCLUDED */ > diff --git a/sql/sql_base.cc b/sql/sql_base.cc > index 0bbcca5..0edda65 100644 > --- a/sql/sql_base.cc > +++ b/sql/sql_base.cc > @@ -1923,7 +1923,9 @@ bool MDL_deadlock_handler::handle_condition(THD *, > */ > > static bool > -open_table_get_mdl_lock(THD *thd, Open_table_context *ot_ctx, > +open_table_get_mdl_lock(THD *thd, > + const DDL_options_st &options, > + Open_table_context *ot_ctx, What's the point - you don't seem to be using options in this function? And that recursively means that you don't need 'options' in all other functions that only pass 'options' argument down to open_table_get_mdl_lock() but otherwise don't use it. > MDL_request *mdl_request, > uint flags, > MDL_ticket **mdl_ticket) > @@ -4176,8 +4182,8 @@ thr_lock_type read_lock_type_for_table(THD *thd, > DBUG_RETURN(FALSE); > > /* Check if CREATE TABLE without REPLACE was used */ > - create_table= (thd->lex->sql_command == SQLCOM_CREATE_TABLE && > - !(thd->lex->create_info.options & HA_LEX_CREATE_REPLACE)); > + create_table= thd->lex->sql_command == SQLCOM_CREATE_TABLE && > + !options.or_replace(); why, options are still in thd->lex, aren't they? > > if (!(flags & MYSQL_OPEN_SKIP_SCOPED_MDL_LOCK)) > { > diff --git a/sql/sql_db.h b/sql/sql_db.h > index 62d379c..dc62e6b 100644 > --- a/sql/sql_db.h > +++ b/sql/sql_db.h > @@ -35,13 +33,178 @@ bool mysql_opt_change_db(THD *thd, > bool my_dboptions_cache_init(void); > void my_dboptions_cache_free(void); > bool check_db_dir_existence(const char *db_name); > -bool load_db_opt(THD *thd, const char *path, HA_CREATE_INFO *create); > +bool load_db_opt(THD *thd, const char *path, Schema_specification_st *create); > bool load_db_opt_by_name(THD *thd, const char *db_name, > - HA_CREATE_INFO *db_create_info); > + Schema_specification_st *db_create_info); > CHARSET_INFO *get_default_db_collation(THD *thd, const char *db_name); > bool my_dbopt_init(void); > void my_dbopt_cleanup(void); > > + > +namespace SQL { > + > + > +namespace Statement { > + > + > +namespace Schema { > + > + > +// Schema name > +class Name: public Identifier_normalized > +{ > +public: > + Name(const LEX_STRING &other) :Identifier_normalized(other) { } > + Name(const LEX *lex) :Identifier_normalized(lex) { } > +#ifdef HAVE_REPLICATION > + bool check_if_slave_ignored(THD *thd) const; > +#endif > + bool check() const; > + bool check_with_error() const; > +}; > + > + > +// Schema definition options: IF EXISTS, IF NOT EXISTS > +class DDL_options: public DDL_options_st > +{ > +public: > + DDL_options() { init(); } > + DDL_options(Options options) > + { > + set(options); > + } > + DDL_options(const DDL_options_st options) > + { > + set(options); > + } > + DDL_options(const LEX *lex); > +}; > + > + > +// Schema specification, e.g. CHARACTER SET xxx > +class Specification: public Schema_specification_st > +{ > +public: > + Specification(const Schema_specification_st &info) > + :Schema_specification_st(info) > + { } > + Specification(const LEX *lex); > +}; > + > + > +/** > + A common class for all Schema DDL options: > + - CREATE SCHEMA > + - DROP SCHEMA > + - ALTER SCHEMA > + - ALTER SCHEMA UPGRADE DATA DIRECTORY NAME > +*/ > +class DDL: public Directly_executable > +{ > +protected: > + bool rm_db(THD *thd, const Name &name, bool if_exists, bool silent) const; > + > + int create_db(THD *thd, > + const Name &name, > + const DDL_options_st options, > + const Schema_specification_st &info, > + bool silent) const; > +}; > + > + > +class DDL_with_name: public DDL, public Name > +{ > + /* > + Execute a DDL statement. > + Assumes that Name has already been checked for validity. > + */ > + virtual bool exec(THD *thd) const = 0; > + virtual bool check_access(THD *) const = 0; > +protected: > + DDL_with_name(const Name &name): Name(name) { } > + DDL_with_name(const LEX *lex): Name(lex) { } > +public: > + int exec_direct(THD *thd) const; > +}; > + > + > +class DDL_with_name_and_options: public DDL_with_name, public DDL_options > +{ > +protected: > + DDL_with_name_and_options(const LEX *lex) > + :DDL_with_name(lex), DDL_options(lex) { } > + DDL_with_name_and_options(const Name &name) > + :DDL_with_name(name), DDL_options() { } > +}; > + > + > +class Create: public DDL_with_name_and_options, public Specification > +{ > + bool exec(THD *thd) const > + { > + return DDL::create_db(thd, *this, *this, *this, false); > + } > + bool check_access(THD *thd) const > + { > + return ::check_access(thd, CREATE_ACL, name(), NULL, NULL, 1, 0); > + } > +public: > + Create(const LEX *lex) > + :DDL_with_name_and_options(lex), Specification(lex) > + { } > + Create(const Name &name, const Schema_specification_st &info) > + :DDL_with_name_and_options(name), Specification(info) > + { } > +}; > + > + > +class Drop: public DDL_with_name_and_options > +{ > + bool exec(THD *thd) const > + { > + return rm_db(thd, *this, if_exists(), false); > + } > + bool check_access(THD *thd) const > + { > + return ::check_access(thd, DROP_ACL, name(), NULL, NULL, 1, 0); > + } > +public: > + Drop(const Name &name) :DDL_with_name_and_options(name) { } > + Drop(const LEX *lex) :DDL_with_name_and_options(lex) { } > +}; > + > + > +class Alter: public DDL_with_name, public Specification > +{ > + bool exec(THD *thd) const; > + bool check_access(THD *thd) const > + { > + return ::check_access(thd, ALTER_ACL, name(), NULL, NULL, 1, 0); > + } > +public: > + Alter(const LEX *lex) :DDL_with_name(lex), Specification(lex) { } > +}; > + > + > +class Upgrade: public DDL_with_name > +{ > + bool exec(THD *thd) const; > + bool check_access(THD *thd) const > + { > + return ::check_access(thd, ALTER_ACL, name(), NULL, NULL, 1, 0) || > + ::check_access(thd, DROP_ACL, name(), NULL, NULL, 1, 0) || > + ::check_access(thd, CREATE_ACL, name(), NULL, NULL, 1, 0); > + } > +public: > + Upgrade(const LEX *lex) :DDL_with_name(lex) { } > + int exec_direct(THD *thd) const; > +}; > + > + > +} // End of namespace SQL::Statement::Schema > +} // End of namespace SQL::Statement > +} // End of namespace SQL Please, no. This and sql_lang.h - make it a separate patch This one should be *only* about splitting HA_CREATE_INFO > + > #define MY_DB_OPT_FILE "db.opt" > > #endif /* SQL_DB_INCLUDED */ > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index b905d8b..c59332e 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -4181,143 +4176,21 @@ static bool do_execute_sp(THD *thd, sp_head *sp) > } > break; > case SQLCOM_CREATE_DB: > - { > - /* > - As mysql_create_db() may modify HA_CREATE_INFO structure passed to > - it, we need to use a copy of LEX::create_info to make execution > - prepared statement- safe. > - */ > - HA_CREATE_INFO create_info(lex->create_info); > - if (check_db_name(&lex->name)) > - { > - my_error(ER_WRONG_DB_NAME, MYF(0), lex->name.str); > - break; > - } > - /* > - If in a slave thread : > - CREATE DATABASE DB was certainly not preceded by USE DB. > - For that reason, db_ok() in sql/slave.cc did not check the > - do_db/ignore_db. And as this query involves no tables, tables_ok() > - above was not called. So we have to check rules again here. > - */ > -#ifdef HAVE_REPLICATION > - if (thd->slave_thread) > - { > - rpl_filter= thd->system_thread_info.rpl_sql_info->rpl_filter; > - if (!rpl_filter->db_ok(lex->name.str) || > - !rpl_filter->db_ok_with_wild_table(lex->name.str)) > - { > - my_message(ER_SLAVE_IGNORED_TABLE, ER(ER_SLAVE_IGNORED_TABLE), MYF(0)); > - break; > - } > - } > -#endif > - if (check_access(thd, CREATE_ACL, lex->name.str, NULL, NULL, 1, 0)) > - break; > - WSREP_TO_ISOLATION_BEGIN(lex->name.str, NULL, NULL) > - res= mysql_create_db(thd, lex->name.str, &create_info, 0); separate refactoring patch, please > + if ((res= SQL::Statement::Schema::Create(lex).exec_direct(thd)) < 0) > + goto error; > break; > - } > case SQLCOM_DROP_DB: > - { > - if (check_db_name(&lex->name)) > - { > - my_error(ER_WRONG_DB_NAME, MYF(0), lex->name.str); > - break; > - } > - /* > - If in a slave thread : > - DROP DATABASE DB may not be preceded by USE DB. > - For that reason, maybe db_ok() in sql/slave.cc did not check the > - do_db/ignore_db. And as this query involves no tables, tables_ok() > - above was not called. So we have to check rules again here. > - */ > -#ifdef HAVE_REPLICATION > - if (thd->slave_thread) > - { > - rpl_filter= thd->system_thread_info.rpl_sql_info->rpl_filter; > - if (!rpl_filter->db_ok(lex->name.str) || > - !rpl_filter->db_ok_with_wild_table(lex->name.str)) > - { > - my_message(ER_SLAVE_IGNORED_TABLE, ER(ER_SLAVE_IGNORED_TABLE), MYF(0)); > - break; > - } > - } > -#endif > - if (check_access(thd, DROP_ACL, lex->name.str, NULL, NULL, 1, 0)) > - break; > - WSREP_TO_ISOLATION_BEGIN(lex->name.str, NULL, NULL) > - res= mysql_rm_db(thd, lex->name.str, lex->check_exists, 0); > + if ((res= SQL::Statement::Schema::Drop(lex).exec_direct(thd)) < 0) > + goto error; > break; > - } > case SQLCOM_ALTER_DB_UPGRADE: > - { > - LEX_STRING *db= & lex->name; > -#ifdef HAVE_REPLICATION > - if (thd->slave_thread) > - { > - rpl_filter= thd->system_thread_info.rpl_sql_info->rpl_filter; > - if (!rpl_filter->db_ok(db->str) || > - !rpl_filter->db_ok_with_wild_table(db->str)) > - { > - res= 1; > - my_message(ER_SLAVE_IGNORED_TABLE, ER(ER_SLAVE_IGNORED_TABLE), MYF(0)); > - break; > - } > - } > -#endif > - if (check_db_name(db)) > - { > - my_error(ER_WRONG_DB_NAME, MYF(0), db->str); > - break; > - } > - if (check_access(thd, ALTER_ACL, db->str, NULL, NULL, 1, 0) || > - check_access(thd, DROP_ACL, db->str, NULL, NULL, 1, 0) || > - check_access(thd, CREATE_ACL, db->str, NULL, NULL, 1, 0)) > - { > - res= 1; > - break; > - } > - WSREP_TO_ISOLATION_BEGIN(db->str, NULL, NULL) > - res= mysql_upgrade_db(thd, db); > - if (!res) > - my_ok(thd); > + if ((res= SQL::Statement::Schema::Upgrade(lex).exec_direct(thd)) < 0) > + goto error; > break; > - } > case SQLCOM_ALTER_DB: > - { > - LEX_STRING *db= &lex->name; > - HA_CREATE_INFO create_info(lex->create_info); > - if (check_db_name(db)) > - { > - my_error(ER_WRONG_DB_NAME, MYF(0), db->str); > - break; > - } > - /* > - If in a slave thread : > - ALTER DATABASE DB may not be preceded by USE DB. > - For that reason, maybe db_ok() in sql/slave.cc did not check the > - do_db/ignore_db. And as this query involves no tables, tables_ok() > - above was not called. So we have to check rules again here. > - */ > -#ifdef HAVE_REPLICATION > - if (thd->slave_thread) > - { > - rpl_filter= thd->system_thread_info.rpl_sql_info->rpl_filter; > - if (!rpl_filter->db_ok(db->str) || > - !rpl_filter->db_ok_with_wild_table(db->str)) > - { > - my_message(ER_SLAVE_IGNORED_TABLE, ER(ER_SLAVE_IGNORED_TABLE), MYF(0)); > - break; > - } > - } > -#endif > - if (check_access(thd, ALTER_ACL, db->str, NULL, NULL, 1, 0)) > - break; > - WSREP_TO_ISOLATION_BEGIN(db->str, NULL, NULL) > - res= mysql_alter_db(thd, db->str, &create_info); > + if ((res= SQL::Statement::Schema::Alter(lex).exec_direct(thd)) < 0) > + goto error; > break; > - } > case SQLCOM_SHOW_CREATE_DB: > { > char db_name_buff[NAME_LEN+1]; > diff --git a/sql/sql_show.cc b/sql/sql_show.cc > index 0541cbc..2536255 100644 > --- a/sql/sql_show.cc > +++ b/sql/sql_show.cc > @@ -1696,14 +1695,14 @@ int show_create_table(THD *thd, TABLE_LIST *table_list, String *packet, > > packet->append(STRING_WITH_LEN("CREATE ")); > if (create_info_arg && > - (create_info_arg->org_options & HA_LEX_CREATE_REPLACE || > + ((create_info_arg->or_replace() && > + !create_info_arg->or_replace_slave_generated()) || that's a bit strange. Could you explain this? why org_options & HA_LEX_CREATE_REPLACE is translated to or_replace() && !or_replace_slave_generated() > create_info_arg->table_was_deleted)) > packet->append(STRING_WITH_LEN("OR REPLACE ")); > if (share->tmp_table) > packet->append(STRING_WITH_LEN("TEMPORARY ")); > packet->append(STRING_WITH_LEN("TABLE ")); > - if (create_info_arg && > - (create_info_arg->options & HA_LEX_CREATE_IF_NOT_EXISTS)) > + if (create_info_arg && create_info_arg->if_not_exists()) > packet->append(STRING_WITH_LEN("IF NOT EXISTS ")); > if (table_list->schema_table) > alias= table_list->schema_table->table_name; > diff --git a/sql/sql_table.cc b/sql/sql_table.cc > index b991215..7bb293b 100644 > --- a/sql/sql_table.cc > +++ b/sql/sql_table.cc > @@ -4628,7 +4628,8 @@ int create_table_impl(THD *thd, > const char *orig_db, const char *orig_table_name, > const char *db, const char *table_name, > const char *path, > - HA_CREATE_INFO *create_info, > + const DDL_options_st options, > + HA_CREATE_INFO &create_info, why? I generally prefer pointers over C++ references - one can immediately see what variables can be modified by the function. is there a compelling reason to use references here? > Alter_info *alter_info, > int create_table_mode, > bool *is_trans, > @@ -5273,9 +5275,10 @@ bool mysql_create_like_table(THD* thd, TABLE_LIST* table, > */ > > /* Copy temporarily the statement flags to thd for lock_table_names() */ > + // QQ: is this really needed??? may be not anymore, as you pass flags separately > uint save_thd_create_info_options= thd->lex->create_info.options; > - thd->lex->create_info.options|= create_info->options; > - res= open_tables(thd, &thd->lex->query_tables, &not_used, 0); > + thd->lex->create_info.options|= create_info.options; > + res= open_tables(thd, create_info, &thd->lex->query_tables, &not_used, 0); > thd->lex->create_info.options= save_thd_create_info_options; > > if (res) > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index 632bdd2..0f2c0e0 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -1621,7 +1623,8 @@ bool my_yyoverflow(short **a, YYSTYPE **b, ulong *yystacksize); > IDENT_sys TEXT_STRING_sys TEXT_STRING_literal > NCHAR_STRING opt_component key_cache_name > sp_opt_label BIN_NUM label_ident TEXT_STRING_filesystem ident_or_empty > - opt_constraint constraint opt_ident opt_if_not_exists_ident > + opt_constraint constraint opt_ident > + opt_if_not_exists_opt_table_element_name can you please try to come up with shorter identifier names? 40 characters - that's a bit too long :) > > %type <lex_str_ptr> > opt_table_alias > diff --git a/storage/xtradb/handler/ha_innodb.cc b/storage/xtradb/handler/ha_innodb.cc > index f3b1167..6b1942e 100644 > --- a/storage/xtradb/handler/ha_innodb.cc > +++ b/storage/xtradb/handler/ha_innodb.cc > @@ -11349,7 +11349,7 @@ static __attribute__((nonnull, warn_unused_result)) > > /* Do not use DATA DIRECTORY with TEMPORARY TABLE. */ > if (create_info->data_file_name > - && create_info->options & HA_LEX_CREATE_TMP_TABLE) { > + && create_info->tmp_table()) { nope, you should not change storage engine API here. tmp_table() was a convenience wrapper, old way should still work > push_warning( > thd, Sql_condition::WARN_LEVEL_WARN, > ER_ILLEGAL_HA_CREATE_OPTION, Regards, Sergei

2 4

[Maria-developers] better execution plan possible with INNER JOIN of mroonga (or whatever storage engine) ?
by Kazuhiko Shiozaki 03 Dec '14

03 Dec '14

Hello ! I am investigating a slow query and have a question about execution plan with mroonga (possibly any storage engine?). (with MariaDB 10.0.15.) Here are tables : CREATE TABLE catalog ( uid bigint(20) unsigned NOT NULL, path varchar(255) COLLATE utf8_unicode_ci NOT NULL DEFAULT '', PRIMARY KEY (uid) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci; CREATE TABLE catalog_full_text ( uid bigint(20) unsigned NOT NULL, title varchar(255) COLLATE utf8_unicode_ci DEFAULT '', PRIMARY KEY (uid), FULLTEXT KEY title (title) COMMENT 'parser "TokenBigramSplitSymbolAlphaDigit"' ) ENGINE=Mroonga DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci; CREATE TABLE category ( uid bigint(20) unsigned NOT NULL, category_uid bigint(20) unsigned NOT NULL DEFAULT '0', base_category_uid bigint(20) unsigned NOT NULL DEFAULT '0', PRIMARY KEY (uid,category_uid,base_category_uid) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci; Here is the query : SELECT DISTINCT catalog.path, catalog.uid FROM ( ( catalog AS catalog INNER JOIN ( category AS category INNER JOIN catalog_full_text AS source_title_full_text ON category.base_category_uid = 18482 AND source_title_full_text.uid = category.category_uid ) ON category.uid = catalog.uid ) INNER JOIN catalog_full_text AS title_full_text ON title_full_text.uid = catalog.uid ) WHERE MATCH (source_title_full_text.title) AGAINST ('bar' IN BOOLEAN MODE) AND MATCH (title_full_text.title) AGAINST ('foo' IN BOOLEAN MODE) Here is the output of EXPLAIN : *************************** 1. row *************************** id: 1 select_type: SIMPLE table: source_title_full_text type: fulltext possible_keys: PRIMARY,title key: title key_len: 0 ref: rows: 1 Extra: Using where with pushed condition; Using temporary *************************** 2. row *************************** id: 1 select_type: SIMPLE table: title_full_text type: fulltext possible_keys: PRIMARY,title key: title key_len: 0 ref: rows: 1 Extra: Using where with pushed condition *************************** 3. row *************************** id: 1 select_type: SIMPLE table: catalog type: eq_ref possible_keys: PRIMARY key: PRIMARY key_len: 8 ref: test.title_full_text.uid rows: 1 Extra: *************************** 4. row *************************** id: 1 select_type: SIMPLE table: category type: eq_ref possible_keys: PRIMARY key: PRIMARY key_len: 24 ref: test.title_full_text.uid,test.source_title_full_text.uid,const rows: 1 Extra: Using index; Distinct As you see, we have two full-text queries at the beginning and both have 'rows = 1'. In a slow query case, 25k records mathces with the first full-text query and 1.6k records matches with the second full-text query. When I manually modify the order of INNER JOIN so that the 'lighter' full-text query appears first, EXPLAIN output is different as well and the performance is much improved. Accoding to Kohei, a Mroonga developer, we always have 'rows = 1' for mroonga full-text query because MariaDB does not request such value to mroonga (JOIN::get_examined_rows() in sql/sql_select.cc), and it is possible to modify mroonga to return an estimation of rows if MariaDB has such an API. So here are my questions : * can MariaDB request an 'estimation of rows' to mroonga (or whatever storage engine) ? * can such information help to decide a better execution plan with INNER JOIN like above ? Thanks in advance ! -- Kazuhiko Shiozaki, Nexedi SA Senior Consultant Nexedi: Consulting and Development of Free / Open Source Software http://www.nexedi.com/ ERP5: Full Featured High End Open Source ERP http://www.erp5.com/

3 3

[Maria-developers] Google's patches to MariaDB are available in the opensource repo
by Pavel Ivanov 03 Dec '14

03 Dec '14

Hi all, Effective immediately the full opensource tree of changes made to MariaDB by Google to make it work for our internal applications is available at https://code.google.com/p/google-mysql/. Note: I didn't update the main page on the site to reflect the changes yet, but the sources are there and the master branch points to the tip of MariaDB tree. Note #2: this tree is not an exact copy of our internal tree, and it's not intended to reflect verbatim our internal development process. It's just a set of changes split roughly one feature per commit with a grouping of several commits related to the same feature set into branches. That means that whenever we find a bug in some of our commits we'll squash the fix in the appropriate place rebuilding the whole git tree on top of that. As a consequence of that links to specific commits will not last forever, though links to specific branches likely will. Note #3: unfortunately code.google.com doesn't have a good visualization of branches in the git tree, so it's not easy to see which commits belong to which branch in our structure of branches stacked on top of each other. So the easiest way to see that is to clone the tree on your local disk and run gitk on it (or other tool that you find more convenient). Besides raising a general awareness of existence of our opensource tree, I'd like to draw attention of MariaDB developers especially to the following branches. I think many commits from there can be considered for inclusion into MariaDB proper. 16-encryption https://code.google.com/p/google-mysql/source/list?name=mariadb-10.0.12%2F1… Encryption of different subsystems writing to disk (not all subsystems included yet, but will be soon). 09-rpl https://code.google.com/p/google-mysql/source/list?name=mariadb-10.0.12%2F0… Replication-related fixes and improvements. 01-base https://code.google.com/p/google-mysql/source/list?name=mariadb-10.0.12%2F0… Mostly basic compilation and testing fixes. 03-privileges https://code.google.com/p/google-mysql/source/list?name=mariadb-10.0.12%2F0… Fixes for some quirks in privilege subsystem (aka sql_acl.cc). If you want to discuss any changes that you see in the tree feel free to reply to this email or start a separate email thread. Thank you, Pavel

2 1

[Maria-developers] Some patches for the C client library
by Daniel Schepler 02 Dec '14

02 Dec '14

While I was creating Debian packages for the "MariaDB Client Library for C" I came across several minor issues that I corrected. I thought I'd post the patches here in case you might be interested in integrating them; I'd have no problem with contributing them under the MCA if that's necessary (i.e. if they're even big enough to require such a thing). Each one has a fairly complete description of its purpose at the start. I've also noticed that mariadb_config --libs prints out something like ... -l-lz if WITH_EXTERNAL_ZLIB=ON. However, for the Debian packages I instead patched mariadb_config to remove the indirect dependencies altogether, as we generally do, so I didn't write a fix for the -l-lz issue. Also, it seems that if I try to build with WITH_SQLITE=ON then I get a build error, so I haven't enabled that in the Debian packages. -- Daniel Schepler

2 2

[Maria-developers] crash, reporting bugs
by Roberto Spadim 02 Dec '14

02 Dec '14

hi guys, i have a consistent crash (mdev-7182) but this email is not about it... when i got the crash, i got this info: ... To report this bug, see http://kb.askmonty.org/en/reporting-bugs ... should we update this url to the current one? https://mariadb.com/kb/en/mariadb/development/debugging-mariadb/reporting-b… thanks -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

3 3

Re: [Maria-developers] [Commits] 020e569: MDEV-4412 : SLOW QUERY LOG - add affected rows (UPDATE / DELETE) in slow query log
by Sergei Golubchik 30 Nov '14

30 Nov '14

Hi, Nirbhay! On Nov 29, nirbhay(a)mariadb.com wrote: > revision-id: 020e569dc7f638b6add8a043b1c9ff308f688ffa > parent(s): 7b55b67de5fdfe91283357fe6c7ccc3f9e355925 > committer: Nirbhay Choubey > branch nick: b4412 > timestamp: 2014-11-29 19:05:32 -0500 > message: > > MDEV-4412 : SLOW QUERY LOG - add affected rows (UPDATE / DELETE) in slow query log > > Added Rows_affected to slow query log & mysql.slow_log table. > > diff --git a/scripts/mysql_system_tables_fix.sql b/scripts/mysql_system_tables_fix.sql > index b9bb59c..aeaf919 100644 > --- a/scripts/mysql_system_tables_fix.sql > +++ b/scripts/mysql_system_tables_fix.sql > @@ -249,7 +249,8 @@ SET GLOBAL general_log = @old_log_state; > SET @old_log_state = @@global.slow_query_log; > SET GLOBAL slow_query_log = 'OFF'; > ALTER TABLE slow_log > - ADD COLUMN thread_id BIGINT(21) UNSIGNED NOT NULL AFTER sql_text; > + ADD COLUMN thread_id BIGINT(21) UNSIGNED NOT NULL AFTER sql_text, > + ADD COLUMN rows_affected INTEGER NOT NULL AFTER thread_id; > ALTER TABLE slow_log > MODIFY start_time TIMESTAMP(6) NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, > MODIFY user_host MEDIUMTEXT NOT NULL, I am not sure it'll work. It would work if you need to upgrade a table that has neither thread_id nor rows_affected column to the latest version. So that ALTER would've been an acceptable solution if both thread_id and rows_affected were added in the same MariaDB version. But they were not and there are quite a few MariaDB releases where slow_log table has thread_id and doesn't have rows_affected. I suspect in that case ALTER will fail. But if you'd like you can try a small test case and see whether such an ALTER will add a second column even if the first one already exists. Better add rows_affected in a separate ALTER TABLE statement. Regards, Sergei

1 0

[Maria-developers] Patch: CREATE [[NO] FORCE] VIEW Options
by James Briggs 30 Nov '14

30 Nov '14

Hi List. Below is my patch that implements the CREATE [[NO] FORCE] VIEW options against MariaDB 10.1.0. Notes: - It's a fairly solid patch, but I think the best thing is to commit it to alpha and let it bake for a while. - it passes t/view.test: # ./mysql-test-run.pl view Logging: ./mysql-test-run.pl view vardir: /usr/local/mariadb-10.1.0/mysql-test/var MariaDB Version 10.1.0-MariaDB-debug TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- main.view [ pass ] 1896 -------------------------------------------------------------------------- The servers were restarted 0 times Spent 1.896 of 7 seconds executing testcases Completed: All 1 tests were successful. - I wrote tests/view.pl which does 8,000 tests. It passes. (I will include it in a separate email.) - Please take a look at the style, and the 411 comments. Thanks, James Briggs. -- Available for MySQL/NoSQL DBA/Programming in San Jose area or remote. $ cat create_force_view.patch --- ../mariadb-10.1.0/sql/sql_view.h 2014-06-27 04:50:36.000000000 -0700 +++ sql/sql_view.h 2014-09-02 02:35:42.000000000 -0700 @@ -29,10 +29,10 @@ /* Function declarations */ bool create_view_precheck(THD *thd, TABLE_LIST *tables, TABLE_LIST *view, - enum_view_create_mode mode); + enum_view_create_mode mode, enum_view_create_force force); bool mysql_create_view(THD *thd, TABLE_LIST *view, - enum_view_create_mode mode); + enum_view_create_mode mode, enum_view_create_force force); bool mysql_make_view(THD *thd, File_parser *parser, TABLE_LIST *table, uint flags); --- ../mariadb-10.1.0/sql/sql_lex.h 2014-06-27 04:50:33.000000000 -0700 +++ sql/sql_lex.h 2014-09-02 01:21:10.000000000 -0700 @@ -170,6 +170,12 @@ VIEW_CREATE_OR_REPLACE // check only that there are not such table }; +enum enum_view_create_force +{ + VIEW_CREATE_NO_FORCE, // default - check that there are not such VIEW/table + VIEW_CREATE_FORCE, // check that there are not such VIEW/table, then ignore table object dependencies +}; + enum enum_drop_mode { DROP_DEFAULT, // mode is not specified @@ -2442,6 +2448,7 @@ }; enum enum_var_type option_type; enum enum_view_create_mode create_view_mode; + enum enum_view_create_force create_view_force; enum enum_drop_mode drop_mode; uint profile_query_id; --- ../mariadb-10.1.0/sql/sql_parse.cc 2014-06-27 04:50:34.000000000 -0700 +++ sql/sql_parse.cc 2014-09-02 02:34:31.000000000 -0700 @@ -4943,7 +4943,7 @@ Note: SQLCOM_CREATE_VIEW also handles 'ALTER VIEW' commands as specified through the thd->lex->create_view_mode flag. */ - res= mysql_create_view(thd, first_table, thd->lex->create_view_mode); + res= mysql_create_view(thd, first_table, thd->lex->create_view_mode, thd->lex->create_view_force); break; } case SQLCOM_DROP_VIEW: --- ../mariadb-10.1.0/sql/sql_yacc.yy 2014-06-27 04:50:37.000000000 -0700 +++ sql/sql_yacc.yy 2014-09-05 17:19:29.000000000 -0700 @@ -1851,7 +1851,7 @@ statement sp_suid sp_c_chistics sp_a_chistics sp_chistic sp_c_chistic xa opt_field_or_var_spec fields_or_vars opt_load_data_set_spec - view_algorithm view_or_trigger_or_sp_or_event + view_algorithm view_or_trigger_or_sp_or_event view_force_option definer_tail no_definer_tail view_suid view_tail view_list_opt view_list view_select view_check_option trigger_tail sp_tail sf_tail udf_tail event_tail @@ -2446,6 +2446,7 @@ VIEW_CREATE_OR_REPLACE); Lex->create_view_algorithm= DTYPE_ALGORITHM_UNDEFINED; Lex->create_view_suid= TRUE; + Lex->create_view_force= VIEW_CREATE_NO_FORCE; /* initialize just in case */ } view_or_trigger_or_sp_or_event { @@ -15887,6 +15888,15 @@ | event_tail ; +view_force_option: + /* empty */ /* 411 - is there a cleaner way of initializing here? */ + { Lex->create_view_force = VIEW_CREATE_NO_FORCE; } + | NO_SYM FORCE_SYM + { Lex->create_view_force = VIEW_CREATE_NO_FORCE; } + | FORCE_SYM + { Lex->create_view_force = VIEW_CREATE_FORCE; } + ; + /************************************************************************** DEFINER clause support. @@ -15944,7 +15954,7 @@ ; view_tail: - view_suid VIEW_SYM table_ident + view_suid view_force_option VIEW_SYM table_ident { LEX *lex= thd->lex; lex->sql_command= SQLCOM_CREATE_VIEW; --- ../mariadb-10.1.0/sql/sql_view.cc 2014-06-27 04:50:36.000000000 -0700 +++ sql/sql_view.cc 2014-09-05 19:33:58.000000000 -0700 @@ -248,7 +248,7 @@ */ bool create_view_precheck(THD *thd, TABLE_LIST *tables, TABLE_LIST *view, - enum_view_create_mode mode) + enum_view_create_mode mode, enum_view_create_force force) { LEX *lex= thd->lex; /* first table in list is target VIEW name => cut off it */ @@ -259,7 +259,7 @@ DBUG_ENTER("create_view_precheck"); /* - Privilege check for view creation: + Privilege check for view creation with default (NO FORCE): - user has CREATE VIEW privilege on view table - user has DROP privilege in case of ALTER VIEW or CREATE OR REPLACE VIEW @@ -272,6 +272,7 @@ checked that we have not more privileges on correspondent column of view table (i.e. user will not get some privileges by view creation) */ + if ((check_access(thd, CREATE_VIEW_ACL, view->db, &view->grant.privilege, &view->grant.m_internal, @@ -285,6 +286,11 @@ check_grant(thd, DROP_ACL, view, FALSE, 1, FALSE)))) goto err; + if (force) { + res = false; + DBUG_RETURN(res || thd->is_error()); + } + for (sl= select_lex; sl; sl= sl->next_select()) { for (tbl= sl->get_table_list(); tbl; tbl= tbl->next_local) @@ -369,7 +375,7 @@ #else bool create_view_precheck(THD *thd, TABLE_LIST *tables, TABLE_LIST *view, - enum_view_create_mode mode) + enum_view_create_mode mode, enum_view_create_force force) { return FALSE; } @@ -391,7 +397,7 @@ */ bool mysql_create_view(THD *thd, TABLE_LIST *views, - enum_view_create_mode mode) + enum_view_create_mode mode, enum_view_create_force force) { LEX *lex= thd->lex; bool link_to_local; @@ -425,14 +431,13 @@ goto err; } - if ((res= create_view_precheck(thd, tables, view, mode))) + if (res= create_view_precheck(thd, tables, view, mode, force)) goto err; lex->link_first_table_back(view, link_to_local); view->open_type= OT_BASE_ONLY; - if (open_temporary_tables(thd, lex->query_tables) || - open_and_lock_tables(thd, lex->query_tables, TRUE, 0)) + if (open_temporary_tables(thd, lex->query_tables) || (!force && open_and_lock_tables(thd, lex->query_tables, TRUE, 0))) { view= lex->unlink_first_table(&link_to_local); res= TRUE; @@ -513,6 +518,7 @@ } } +if (!force) { /* prepare select to resolve all fields */ lex->context_analysis_only|= CONTEXT_ANALYSIS_ONLY_VIEW; if (unit->prepare(thd, 0, 0)) @@ -612,6 +618,7 @@ } } #endif +} res= mysql_register_view(thd, view, mode); @@ -621,7 +628,7 @@ meta-data changes after ALTER VIEW. */ - if (!res) + // if (!res) + if (!res && !force) /* 411 - solves segfault problems with CREATE FORCE VIEW option sometimes */ tdc_remove_table(thd, TDC_RT_REMOVE_ALL, view->db, view->table_name, false); if (mysql_bin_log.is_open()) @@ -908,6 +915,8 @@ fn_format(path_buff, file.str, dir.str, "", MY_UNPACK_FILENAME); path.length= strlen(path_buff); if (ha_table_exists(thd, view->db, view->table_name, NULL)) { if (mode == VIEW_CREATE_NEW) --- ../mariadb-10.1.0/mysql-test/t/view.test 2014-06-27 04:50:30.000000000 -0700 +++ mysql-test/t/view.test 2014-09-06 00:23:32.000000000 -0700 @@ -5263,4 +5263,17 @@ --echo # ----------------------------------------------------------------- --echo # -- End of 10.0 tests. --echo # ----------------------------------------------------------------- + +create no force view v1 as select 1; +drop view if exists v1; + +create force view v1 as select 1; +drop view if exists v1; + +create force view v1 as select * from missing_base_table; +drop view if exists v1; + +--echo # ----------------------------------------------------------------- +--echo # -- End of 10.1 tests. +--echo # ----------------------------------------------------------------- SET optimizer_switch=@save_optimizer_switch;

2 2

Re: [Maria-developers] MDEV-4412 : SLOW QUERY LOG - add affected rows (UPDATE / DELETE) in slow query log
by Sergei Golubchik 30 Nov '14

30 Nov '14

Hi, Nirbhay! On Nov 28, nirbhay(a)mariadb.com wrote: > revision-id: 7a5559421f4f3dda44d04bf946e79661d09b4dc2 > parent(s): 7b55b67de5fdfe91283357fe6c7ccc3f9e355925 > committer: Nirbhay Choubey > branch nick: b4412 > timestamp: 2014-11-28 20:53:02 -0500 > message: > > MDEV-4412 : SLOW QUERY LOG - add affected rows (UPDATE / DELETE) in slow query log > > Added Rows_affected to slow query log & mysql.slow_log table. Just a couple of comments: > diff --git a/scripts/mysql_system_tables_fix.sql b/scripts/mysql_system_tables_fix.sql > index b9bb59c..4973dc9 100644 > --- a/scripts/mysql_system_tables_fix.sql > +++ b/scripts/mysql_system_tables_fix.sql > @@ -262,7 +262,8 @@ ALTER TABLE slow_log > MODIFY insert_id INTEGER NOT NULL, > MODIFY server_id INTEGER UNSIGNED NOT NULL, > MODIFY sql_text MEDIUMTEXT NOT NULL, > - MODIFY thread_id BIGINT(21) UNSIGNED NOT NULL; > + MODIFY thread_id BIGINT(21) UNSIGNED NOT NULL, > + MODIFY rows_affected INTEGER NOT NULL; Eh? That makes no sense. This file is for mysql_upgrade. Your ALTER TABLE would help if there was some MariaDB version where rows_affected column would've been nullable and you wanted to change it to NOT NULL. But there was no such MariaDB version. There was MariaDB version without rows_affected column at all. So, instead of this MODIFY rows_affected, you need to put another ALTER TABLE statement that adds rows_affected INTEGER NOT NULL column. > SET GLOBAL slow_query_log = @old_log_state; > > ALTER TABLE plugin > diff --git a/sql/log.cc b/sql/log.cc > index ff04105..24e5283 100644 > --- a/sql/log.cc > +++ b/sql/log.cc > @@ -2918,12 +2924,16 @@ bool MYSQL_QUERY_LOG::write(THD *thd, time_t current_time, > sprintf(lock_time_buff, "%.6f", ulonglong2double(lock_utime)/1000000.0); > if (my_b_printf(&log_file, > "# Thread_id: %lu Schema: %s QC_hit: %s\n" \ > - "# Query_time: %s Lock_time: %s Rows_sent: %lu Rows_examined: %lu\n", > + "# Query_time: %s Lock_time: %s Rows_sent: %lu Rows_examined: %lu\n" \ > + "# Rows_affected: %lu\n", I'd rather have Rows_affected on the same line with Rows_sent and Rows_examined > (ulong) thd->thread_id, (thd->db ? thd->db : ""), > ((thd->query_plan_flags & QPLAN_QC) ? "Yes" : "No"), > query_time_buff, lock_time_buff, > (ulong) thd->get_sent_row_count(), > - (ulong) thd->get_examined_row_count()) == (size_t) -1) > + (ulong) thd->get_examined_row_count(), > + thd->get_stmt_da()->is_ok() ? > + (ulong) thd->get_stmt_da()->affected_rows() : > + 0) == (size_t) -1) > tmp_errno= errno; > if ((thd->variables.log_slow_verbosity & LOG_SLOW_VERBOSITY_QUERY_PLAN) && > (thd->query_plan_flags & Regards, Sergei

2 1

Re: [Maria-developers] [Commits] Rev 4484: MDEV-4255 MIXED replication: Unique key updates are not marked unsafe, replication fails in lp:~maria-captains/maria/10.0
by Kristian Nielsen 29 Nov '14

29 Nov '14

Sergei Golubchik <serg(a)mariadb.org> writes: > Kristian Nielsen, could you please review this patch? I guess my main question here is, did you think about why you want to) fix this 1) in a post-GA release, 2) at all? One problem with adding new statements as unsafe for statement-based binlogging is that it introduces new warnings in existing applications. This is especially annoying on a slave with binlogging enabled, as every unsafe statement now causes a warning in the slave error log. Thus, it can happen that a security update for a user leaves them with an error log that constantly fills up on a busy slave. It is my understanding that statement-based binlogging (as opposed to mixed or row) is still quite popular, and there have been complaints about these warnings before. Another point is that this patch seems to fix only a corner case of a bigger problem? This particular patch is about non-transactional updates. But updating a unique key can break replication just as well on transactional tables: CREATE TABLE t1 (a INT PRIMARY KEY); INSERT INTO t1 VALUES (2), (3), (4); UPDATE t1 SET a=a-1; Since we do not have deferred constraints, this UPDATE will fail or succeed depending on the order in which rows are updated. So a statement that succeeds andgets binlogged on the master can break on the slave if the rows are updated in a different order. So any reason that you fix one part of UPDATE of unique key but not the other? (I suppose the same problem can also happen with foreign key constraints, though it might take some creativity to come up with an example where an UPDATE will conflict or not with foreign keys depending on row update order). The bug has fix version 5.5, your patch seems to be against 10.0, where do you intend to push it? I would probably have chosen to fix in 10.1, if at all, but I'm not necessarily against 5.5 or 10.0, I'm mostly just asking what the reason is for whichever version was chosen? > I don't particularly like that it explicitly invokes > thd->decide_logging_format() . But there's no way around it, first time > logging format is decided in lock_tables(), basically when tables are just > opened. At that point we don't know what fields will be updated yet. I am not really familiar with how binlogging is decided on the master, have not worked with that code so far. So unfortunately I am not able to say much about whether this is a problem or not. At least the call is still before the actual update of rows starts, so I suppose it should be ok... > At lp:~maria-captains/maria/10.0 > > ------------------------------------------------------------ > revno: 4484 > revision-id: sergii(a)pisem.net-20141116234900-747qdhm9ti9xh070 > parent: sergii(a)pisem.net-20141113161325-mdz5ox7e86iyazss > fixes bug: https://mariadb.atlassian.net/browse/MDEV-4255 > committer: Sergei Golubchik <sergii(a)pisem.net> > branch nick: 10.0 > timestamp: Mon 2014-11-17 00:49:00 +0100 > message: > MDEV-4255 MIXED replication: Unique key updates are not marked unsafe, replication fails I think this needs a better description, especially since this can cause new warnings in slave error logs. Here is a suggestion, but feel free to use something else if you prefer: "MDEV-4255 MIXED replication: Unique key updates are not marked unsafe, replication fails Statement-based replication of UPDATE statements that modify (parts of) unique keys is not safe. If the order of row updates differ between master and slave, the slave can end up modifying a different set of rows before a unique key violation error than what was updated on the master. If the table is non-transactional, then the statement will be partially executed on the master and on the slave, but with different result, causing replication to diverge. This patch marks as unsafe for statement-based replication any UPDATE that modifies a unique key on a non-transactinal table. In mixed-mode replication, this will cause the query to be logged in row mode. In statement-mode replication, this will produce a warning about binlogging of unsafe statement." > === modified file 'sql/share/errmsg-utf8.txt' > --- a/sql/share/errmsg-utf8.txt 2014-11-13 16:13:25 +0000 > +++ b/sql/share/errmsg-utf8.txt 2014-11-16 23:49:00 +0000 > @@ -7111,3 +7111,5 @@ ER_SLAVE_SKIP_NOT_IN_GTID > eng "When using GTID, @@sql_slave_skip_counter can not be used. Instead, setting @@gtid_slave_pos explicitly can be used to skip to after a given GTID position." > ER_TABLE_DEFINITION_TOO_BIG > eng "The definition for table %`s is too big" > +ER_BINLOG_UNSAFE_UPDATE_UNIQUE_NONTRANS > + eng "The statement is unsafe because it updates a UNIQUE key colums in a non-transactional table. This is unsafe because in the case of a conflict a non-predictable part of the table will stay updated." (Adding a new error code in a GA release requires some care, especially if fix-version:5.5 is intended, but I'm sure you're better aware of this than most). > === modified file 'sql/sql_lex.h' > --- a/sql/sql_lex.h 2014-08-07 16:06:56 +0000 > +++ b/sql/sql_lex.h 2014-11-16 23:49:00 +0000 > @@ -1319,6 +1319,12 @@ class Query_tables_list > */ > BINLOG_STMT_UNSAFE_AUTOINC_NOT_FIRST, > > + /** > + INSERT into auto-inc field which is not the first part of composed > + primary key. > + */ > + BINLOG_STMT_UPDATE_UNIQUE_NONTRANS, > + Copy-paste error. Here, you copied the comment of the earlier entry, you surely intended to re-write the comment to describe the newly added entry. Hope this helps, - Kristian.

3 7

Re: [Maria-developers] [Commits] 953dd8d: MDEV-6997 MDEV-6996 MDEV-7002 MDEV-7006 MDEV-7009 MDEV-7012 MDEV-7015 MDEV-6977
by Sergei Golubchik 29 Nov '14

29 Nov '14

Hi, sanja! On Nov 05, sanja(a)mariadb.com wrote: > revision-id: 953dd8d87fea224e26d11452eb458c79ba098f4a > parent(s): d6981391eb8f904135dfdedb5b90a7592b35e648 > committer: Oleksandr Byelkin > branch nick: work-maria-10.1-sysvar4 > timestamp: 2014-11-05 11:05:01 +0100 > message: > > MDEV-6997 MDEV-6996 MDEV-7002 MDEV-7006 MDEV-7009 MDEV-7012 MDEV-7015 MDEV-6977 I'd prefer a slightly more verbose commit comment. And don't forget - first line is the subject. For example: Use NO_SET_STMT flag for more variables Fixes MDEV-6997 MDEV-6996 MDEV-7002 MDEV-7006 MDEV-7009 MDEV-7012 MDEV-7015 MDEV-6977 I won't comment on individual variables here, as you're simply disabling them all. See my comments in Jira. Regards, Sergei

1 0

[Maria-developers] MDEV-7004 review
by Sergei Golubchik 28 Nov '14

28 Nov '14

Hi, Sergey I didn't review MDEV-7200 (InnoDB) and patches that are already in 10.1. Almost everything else was fine, see few comments below. > ------------------------------------------------------------ > revno: 4513 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0-power > timestamp: Fri 2014-11-07 22:03:18 +0400 > message: > Multi-rwlock lock for table definition cache hash. > added: > modified: > diff: > === modified file 'sql/table_cache.cc' > --- sql/table_cache.cc 2014-10-21 12:20:39 +0000 > +++ sql/table_cache.cc 2014-11-07 18:03:18 +0000 > @@ -53,6 +53,77 @@ > #include "table.h" > #include "sql_base.h" > > + > +/** > + Table definition hash lock. > + > + Lock consists of multiple rw-locks (8 by default). Lock is write locked > + when all instances are write locked. Lock is read locked when any instance > + is read locked. > + > + This class was implemented to workaround scalability bottleneck of table > + definition hash rw-lock. > + > + To be replaced with lock-free hash when we understand how to solve the > + following problems: > + - lock-free hash iterator (can be workarounded by introducing global > + all_tables list); I don't particularly like that solution. Could you try to switch to lock-free hash now? I'll try to do the iterator asap > + - lf_hash_search() may fail because of OOM. It is rather complex to handle > + it properly. > +*/ > + > +class TDC_lock > +{ > + uint m_instances; > + bool m_write_locked; > + char pad[128]; > + struct st_locks > + { > + mysql_rwlock_t lock; > + char pad[128]; In these cases I usually do: pad[128 - sizeof(mysql_rwlock_t)]; > + } *locks; > +public: > + int init(PSI_rwlock_key key, uint instances) > + { > + m_instances= instances; > + m_write_locked= false; > + locks= (st_locks *) my_malloc(sizeof(struct st_locks) * m_instances, MYF(MY_WME)); > + if (!locks) > + return 1; > + for (uint i= 0; i < m_instances; i++) > + mysql_rwlock_init(key, &locks[i].lock); > + return 0; > + } > + void destroy(void) > + { > + for (uint i= 0; i < m_instances; i++) > + mysql_rwlock_destroy(&locks[i].lock); > + my_free(locks); > + } > + void wrlock(void) > + { > + for (uint i= 0; i < m_instances; i++) > + mysql_rwlock_wrlock(&locks[i].lock); > + m_write_locked= true; > + } > + void rdlock(THD *thd= 0) > + { > + mysql_rwlock_rdlock(&locks[thd ? thd->thread_id % m_instances : 0].lock); > + } > + void unlock(THD *thd= 0) > + { > + if (m_write_locked) > + { > + m_write_locked= false; > + for (uint i= 0; i < m_instances; i++) > + mysql_rwlock_unlock(&locks[i].lock); > + } > + else > + mysql_rwlock_unlock(&locks[thd ? thd->thread_id % m_instances : 0].lock); > + } > +}; > + > + > /** Configuration. */ > ulong tdc_size; /**< Table definition cache threshold for LRU eviction. */ > ulong tc_size; /**< Table cache threshold for LRU eviction. */ > ------------------------------------------------------------ > revno: 4512 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0-power > timestamp: Fri 2014-10-31 15:44:36 +0400 > message: > Preallocate locks on THD mem_root to avoid expensive malloc. > modified: > diff: > === modified file 'sql/lock.cc' > --- sql/lock.cc 2014-09-30 17:31:14 +0000 > +++ sql/lock.cc 2014-10-31 11:44:36 +0000 > @@ -700,7 +699,7 @@ MYSQL_LOCK *get_lock_data(THD *thd, TABL > TABLE **to, **table_buf; > DBUG_ENTER("get_lock_data"); > > - DBUG_ASSERT((flags == GET_LOCK_UNLOCK) || (flags == GET_LOCK_STORE_LOCKS)); > + DBUG_ASSERT((flags & GET_LOCK_UNLOCK) || (flags & GET_LOCK_STORE_LOCKS)); Not the same, old assert also verified that these flags aren't set both. New assert doesn't. But I don't think it matters much. > DBUG_PRINT("info", ("count %d", count)); > > for (i=lock_count=table_count=0 ; i < count ; i++) > ------------------------------------------------------------ > revno: 4508 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0 > timestamp: Tue 2014-10-21 16:20:39 +0400 > message: > modified: > diff: > === modified file 'sql/handler.cc' > --- sql/handler.cc 2014-11-13 10:01:31 +0000 > +++ sql/handler.cc 2014-10-21 12:20:39 +0000 > @@ -410,13 +410,16 @@ static int hton_ext_based_table_discover > static void update_discovery_counters(handlerton *hton, int val) > { > if (hton->discover_table_existence == full_discover_for_existence) > - my_atomic_add32(&need_full_discover_for_existence, val); > + my_atomic_add32_explicit(&need_full_discover_for_existence, val, > + MY_MEMORY_ORDER_RELAXED); > > if (hton->discover_table_names) > - my_atomic_add32(&engines_with_discover_table_names, val); > + my_atomic_add32_explicit(&engines_with_discover_table_names, val, > + MY_MEMORY_ORDER_RELAXED); > > if (hton->discover_table) > - my_atomic_add32(&engines_with_discover, val); > + my_atomic_add32_explicit(&engines_with_discover, val, > + MY_MEMORY_ORDER_RELAXED); I wouldn't do that. Technically there's a dependency between these counters and loaded engines, so the order, kind of, matters. I don't think there is any practical risk of bugs here, but, on the other hand, these counters are only updated when an engine is loaded or unloaded, so there's no need to bother with the relaxed memory ordering here. > } > > int ha_finalize_handlerton(st_plugin_int *plugin) > > === modified file 'sql/mysqld.cc' > --- sql/mysqld.cc 2014-11-18 21:25:47 +0000 > +++ sql/mysqld.cc 2014-10-21 12:20:39 +0000 > @@ -3884,7 +3884,7 @@ static void my_malloc_size_cb_func(long > } > // workaround for gcc 4.2.4-1ubuntu4 -fPIE (from DEB_BUILD_HARDENING=1) > int64 volatile * volatile ptr=&global_status_var.memory_used; > - my_atomic_add64(ptr, size); > + my_atomic_add64_explicit(ptr, size, MY_MEMORY_ORDER_RELAXED); good! > } > } > > > === modified file 'sql/mysqld.h' > --- sql/mysqld.h 2014-11-13 08:56:28 +0000 > +++ sql/mysqld.h 2014-10-21 12:20:39 +0000 > @@ -629,7 +629,7 @@ inline __attribute__((warn_unused_result > { > query_id_t id; > my_atomic_rwlock_wrlock(&global_query_id_lock); > - id= my_atomic_add64(&global_query_id, 1); > + id= my_atomic_add64_explicit(&global_query_id, 1, MY_MEMORY_ORDER_RELAXED); I believe this is fine > my_atomic_rwlock_wrunlock(&global_query_id_lock); > return (id); > } > === modified file 'sql/table_cache.cc' > --- sql/table_cache.cc 2014-06-10 18:20:33 +0000 > +++ sql/table_cache.cc 2014-10-21 12:20:39 +0000 > @@ -139,7 +139,7 @@ uint tc_records(void) > { > uint count; > my_atomic_rwlock_rdlock(&LOCK_tdc_atomics); > - count= my_atomic_load32(&tc_count); > + count= my_atomic_load32_explicit(&tc_count, MY_MEMORY_ORDER_RELAXED); I'm not sure about this file. You should know better whether it's safe to use relaxed memory ordering here. > my_atomic_rwlock_rdunlock(&LOCK_tdc_atomics); > return count; > } Regards, Sergei

2 5

[Maria-developers] add a rotate event at begin of relay log make greate help
by nanyi607rao 28 Nov '14

28 Nov '14

Hi kristian, currently, a relaylog is immediately purged after it be completely replayed (relay_log_purge=1). and if IO thread crashes, we should delete all relay logs and fetch master binlog again from exec_master_log_pos/file (relay_log_purge=ON). I think that make a greap waste of disk IO especially to cloud disk. The critical problem is that there is no method to know the original binlog filename on master of a event in relaylog. Actually we can know a relaylog event's original binlog offset on master, but we can't know the original binlog filename. so why not forcely add a rotate event or any other new type event which contained the original master binlog filename at begin of a relay log. as I know two different master binlogs' events wouldn't contained in one relay log, so the original binlog filename of a relaylog's events is only one. If we want to know a relaylog event's original binlog filename/position, we can get original filename from the begin of relaylog, and get original binlog offset from event's "end log pos". oppositely, if we know a event's binlog filename/end positon on master, wo also can quickly find it in slave's relay log. That can make many benefits, firstly no need to delete all relay logs and fetch master binlogs again when IO thread crashed, because we can get exact last read_master_log_filename/position from last relay log. secondly, there is no need to use GTID in 1 vs n replication failover Scenario (Gtid must set log_slave_updates=ON in mysql 5.6, which increase disk IO load), if master crashes, other slaves can get lost events for the newest slave's relay log, as long as relay log don't be purged, then promotes the newest slave to master. Thanks. 2014-11-26 nanyi607rao

3 3

[Maria-developers] MDEV-6431 review
by Sergey Vojtovich 27 Nov '14

27 Nov '14

Hi Sergei, overall it looks good. I want to double check the clean-up stuff, since I didn't understand everything at the first glance. Some comments inline. > diff --git a/include/mysql/plugin_password_validation.h b/include/mysql/plugin_password_validation.h > new file mode 100644 > index 0000000..0bbb8f4 > --- /dev/null > +++ b/include/mysql/plugin_password_validation.h > @@ -0,0 +1,45 @@ > +/** > + Password validation plugin descriptor > +*/ > +struct st_mysql_password_validation > +{ > + int interface_version; /**< version plugin uses */ > + /** > + Function provided by the plugin which should perform password validation > + and return 0 if the password has passed the validation. > + */ > + int (*validate_password)(MYSQL_LEX_STRING *username, > + MYSQL_LEX_STRING *password); > +}; > +#endif > + This differs from MySQL API, so we can't reuse their plugins. Also they have this cool get_password_strength method, but that's something we can expose via UDF. OTOH we can compare password against username, and that's quite common password misuse. And what I always hated in this validation stuff is that you'll never come up with a password that makes it happy. Of course unless you know requirements. In our case requirements are exposed via system variables. This is more or less acceptable. But I believe it would be nice if this stuff would be able to generate strong passwords. Again, that's something that can be exposed as UDF. > diff --git a/sql/sql_plugin.cc b/sql/sql_plugin.cc > index 529a795..cdf8c7b 100644 > --- a/sql/sql_plugin.cc > +++ b/sql/sql_plugin.cc > @@ -1051,7 +1055,8 @@ static bool plugin_add(MEM_ROOT *tmp_root, > continue; // invalid plugin type > > if (plugin->type == MYSQL_UDF_PLUGIN || > - (plugin->type == 8 && tmp.plugin_dl->mariaversion == 0)) > + (plugin->type == MariaDB_PASSWORD_VALIDATION_INTERFACE_VERSION && > + tmp.plugin_dl->mariaversion == 0)) > continue; // unsupported plugin type > > if (name->str && my_strnncoll(system_charset_info, You compare type against version here. Why? Regards, Sergey

2 1

Re: [Maria-developers] [Commits] Rev 4369: MDEV-7148 - Recurring: InnoDB: Failing assertion: !lock->recursive in lp:maria/5.5
by Kristian Nielsen 27 Nov '14

27 Nov '14

Sergey Vojtovich <svoj(a)mariadb.org> writes: > revno: 4369 > revision-id: svoj(a)mariadb.org-20141120094823-fozdsrm1kzv46kxi > parent: jplindst(a)mariadb.org-20141119182734-cwbaed0ka90ocj5e > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 5.5 > timestamp: Thu 2014-11-20 13:48:23 +0400 > message: > MDEV-7148 - Recurring: InnoDB: Failing assertion: !lock->recursive > > On PPC64 high-loaded server may crash due to assertion failure in InnoDB > rwlocks code. > > This happened because load order between "recursive" and "writer_thread" > wasn't properly enforced. Looks ok. I failed to understand why the patch would fix the assertion mentioned in the bug report. But the patch itself looks correct. - Kristian.

3 3

Re: [Maria-developers] [Commits] Rev 3902: MDEV-7123: MariaDB 10.0.14 Galera node shutdown with signal 11 in lp:~maria-captains/maria/maria-10.0-galera
by Kristian Nielsen 26 Nov '14

26 Nov '14

Nirbhay Choubey <nirbhay(a)mariadb.com> writes: > revno: 3902 > revision-id: nirbhay(a)mariadb.com-20141123163533-9ov1yirj26cur301 > parent: nirbhay(a)mariadb.com-20141119183337-n5spskf54beqbhif > committer: Nirbhay Choubey <nirbhay(a)mariadb.com> > branch nick: maria-10.0-galera > timestamp: Sun 2014-11-23 11:35:33 -0500 > message: > MDEV-7123: MariaDB 10.0.14 Galera node shutdown with signal 11 > > wsrep-patch uses same connection name for constructing Master_info > objects. As a result all existing wsrep Master_info objects refer > to same rpl_filter object. This could lead to race when multiple > threads try to destruct/delete Master_info object, as they would all > try to delete the same relay_log object. > Fixed by adding a mutex to protect the critical section. The patch is ok. But please consider the following points, and act on them or not as you see fit: 1. You are introducing a mutex lock/unlock in a general data structure used in a couple of places (looks like keycaches and rpl_filters?). Did you consider the performance impact of this? When will the additional locking/unlocking take place? If it only happens in less often used operations (like re-configuring replication filters or key caches?), then extra locking is not an issue. If it happens often, like once per query or something like that, it would probably be best to find a solution with less performance impact. (mutex lock and unlock are quite expensive even if not contended). 2. Consider using explicit mysql_mutex_lock() and mysql_mutex_unlock() calls rather than introducing extra lock()/unlock() methods. The extra methods are needless abtractions that make it harder to understand exactly what is happening wrt. locking. 3. Did you consider if a test case could be made for the originally reported bug? We used to have a rule that any bug fix must include a test case that reproduces the bug fixed (ie. the test case must fail before the patch). Not a hard rule (a few bugs can be extremely hard to write mtr testcases for), but a good rule nevertheless. But maybe this rule has been dropped, now that the focus is on quantity of Jira tickes closed rather than quality. - Kristian.

1 0

[Maria-developers] Windows: git config autocrlf
by Alexander Barkov 26 Nov '14

26 Nov '14

Hi, What would be the correct value for the autocrlf setting in git on Windows? Git suggests autocrlf=true by default at installation time. But then mysql-test do not work with this value. What is a better choice between the other two: "false" or "input"? Thanks.

1 0

[Maria-developers] doubt about read uncommited
by Roberto Spadim 25 Nov '14

25 Nov '14

hi guys, i use read uncommited sometimes with innodb, that's nice and work but now, i'm using a myisam table, and a aria table.... does read uncommited work with this kind of engine? i tested and table stay 'waiting table lock' while a looooong update occurs i don't know what's the internal diference but is possible to "easily" implement read uncommited to myisam/aria? -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

4 10

[Maria-developers] Please review: MDEV-7199 Introduce the concept of data type handlers
by Alexander Barkov 25 Nov '14

25 Nov '14

Hi Sergei, while working on addressing your review suggestions for "MDEV-4912 Add a plugin to field types (column types)", I decided to create a separate tasks for a piece of additional changes: https://mariadb.atlassian.net/browse/MDEV-7199 MDEV-7199 Introduce the concept of data type handlers Please review. Thanks.

1 0

Re: [Maria-developers] [Commits] Rev 4507: MDEV-7157 plugins.server_audit fails sporadically in buildbot. in file:///home/hf/wmar/mdev-7157/
by Kristian Nielsen 24 Nov '14

24 Nov '14

<holyfoot(a)askmonty.org> writes: > ------------------------------------------------------------ > revno: 4507 > revision-id: holyfoot(a)askmonty.org-20141123225345-p2xd7yrklzee10u1 > parent: sergii(a)pisem.net-20141121192039-d0lv6cj96kg5pw02 > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: mdev-7157 > timestamp: Mon 2014-11-24 02:53:45 +0400 > message: > MDEV-7157 plugins.server_audit fails sporadically in buildbot. > Records can get to the different place in the log when multiple thread > are logged. So the delay added to let the record be saved on the same > place. > === modified file 'mysql-test/suite/plugins/t/server_audit.test' > --- a/mysql-test/suite/plugins/t/server_audit.test 2014-03-26 19:58:27 +0000 > +++ b/mysql-test/suite/plugins/t/server_audit.test 2014-11-23 22:53:45 +0000 > @@ -43,6 +43,7 @@ show variables like 'server_audit%'; > set global server_audit_mode=1; > set global server_audit_events=''; > create database sa_db; > +--sleep 2 The correct way to do this is to use DEBUG_SYNC... - Kristian.

1 0

Re: [Maria-developers] [Commits] Rev 3879: MDEV-6593 : domain_id based replication filters in lp:~maria-captains/maria/maria-10.0-galera
by Kristian Nielsen 24 Nov '14

24 Nov '14

Nirbhay Choubey <nirbhay(a)skysql.com> writes: > ------------------------------------------------------------ > revno: 3879 > revision-id: nirbhay(a)skysql.com-20140912181622-s33e7r0vtgpju95q > parent: nirbhay(a)skysql.com-20140814224304-rtkeal7p9lk06li1 > committer: Nirbhay Choubey <nirbhay(a)skysql.com> > branch nick: maria-10.0-galera_6593 > timestamp: Fri 2014-09-12 14:16:22 -0400 > message: > MDEV-6593 : domain_id based replication filters > > Added support for IGNORE_DOMAIN_IDS and DO_DOMAIN_IDS. Here is my review. Some general comments, followed by some detailed comments in-line in the patch: 1. Did you prepare any documentation? You could either add it to the knowledgebase directly, or write up a text file and ask Daniel to add it in relevant places. 2. Did you run the full test suite? I would have expected at least some tests in the multi_source suite to need modification. The chance touches enough things that you should test it in a feature tree in Buildbot to ensure that it is green before pushing it to a main tree. 3. I think some extra test cases are needed: - You should test (and document) what happens when the same domain_id is in both the DO and the IGNORE list. Is it actually ever sensible to have both DO_DOMAIN_IDS and IGNORE_DOMAIN_IDS ? If not, I think it should be an error to try. - You should also add a test using non-GTID mode. - You should also add a test where parallel replication is used (--slave-parallel-threads > 0). - Add a test for the following scenario when using GTID mode: Some domain id is ignored, some events are received from the master and ignored. Then the slave is stopped, and the ignore rule is removed. What happens when the slave is restarted and reconnects? I think the previously ignored domain id events will now be re-fetched and replicated. I guess that's ok, but it should be documented. Also test what happens when binlog files on the master have been rotated and purged, so that the events are no longer available on the master (an error should occur). - Add a similar test for when not using GTID mode, the last event replicated by the slave is ignored, the slave is stopped, the ignore rule removed and the slave restarted; will the last event now be re-fetched or not (I think not). - Add a test that checks that seconds_behind_master is properly updated to zero even when the last event is ignored due to DO_DOMAIN_IDS/IGNORE_DOMAIN_IDS. It can perhaps be a bit tricky to test, but I think you can use this existing test as a basis: mysql-test/suite/rpl/t/rpl_parallel2.test - It would also be good to check that the code works when used on an old master (which does not send GTID events). You can do this by creating a binlog file with a 5.5 server and storing it in mysql-test/std_data/, and then installing it in a master; there should be several other test cases that already do this. 4. Also see detailed comments for some possible problems with the implementation. The most serious is probably to ensure that events are not skipped after the end of the group, we need a couple of tests for this, see below. Also, there is a small memory leak, see below. > +DO_DOMAIN_IDS (before) : 1 > +IGNORE_DOMAIN_IDS (before) : 2 > +CHANGE MASTER TO DO_DOMAIN_IDS=(4,4,5,1,7,7,7,1,1,2,6,8,1,4,5,5,9,3), MASTER_USE_GTID=current_pos; So if a domain_id is specified multiple times, the redundant ones will be ignored. It would seem preferable to give an error... but as far as I can see the behaviour is the same for IGNORE_SERVER_IDS, so better to be consistent as you did in the code. Just be sure to add this to the documentation. > === added file 'mysql-test/suite/rpl/t/rpl_domain_id_filter.test' > --- a/mysql-test/suite/rpl/t/rpl_domain_id_filter.test 1970-01-01 00:00:00 +0000 > +++ b/mysql-test/suite/rpl/t/rpl_domain_id_filter.test 2014-09-12 18:16:22 +0000 > +SET @gtid_domain_id_saved = @@session.gtid_domain_id; This is not needed, nor do you use it anywhere, you can remove it. > +##### Case 3: When both DO_DOMAIN_IDS and IGNORE_DOMAIN_IDS are non-empty > + > +source include/stop_slave.inc; > +let $do_domain_ids_before= query_get_value(SHOW SLAVE STATUS, Replicate_Do_Domain_Ids, 1); > +let $ignore_domain_ids_before= query_get_value(SHOW SLAVE STATUS, Replicate_Ignore_Domain_Ids, 1); > +--echo DO_DOMAIN_IDS (before) : $do_domain_ids_before > +--echo IGNORE_DOMAIN_IDS (before) : $ignore_domain_ids_before > + > +# Replicate events belonging to "domain_id 1". (DO_DOMAIN_IDS should win!) > +CHANGE MASTER TO DO_DOMAIN_IDS=(1), IGNORE_DOMAIN_IDS=(2), MASTER_USE_GTID=current_pos; Did you intend here to test overlapping ids in DO_DOMAIN_IDS and IGNORE_DOMAIN_IDS (due to the comment "DO_DOMAIN_IDS should win") ? But if I understand correctly, IGNORE_DOMAIN_IDS will be ignored if DO_DOMAIN_IDS is non-empty. So I think it should be an error to try to set both - do you agree? > +--source include/rpl_end.inc Please add additional testcases, as suggested at the start of review. > === modified file 'sql/rpl_mi.cc' > --- a/sql/rpl_mi.cc 2014-02-19 10:05:15 +0000 > +++ b/sql/rpl_mi.cc 2014-09-12 18:16:22 +0000 > @@ -513,6 +513,25 @@ > else > mi->using_gtid= Master_info::USE_GTID_NO; > } > + > + // DO_DOMAIN_IDS > + if (0 == strncmp(buf, STRING_WITH_LEN("do_domain_ids=")) && > + mi->domain_id_filter.init_ids(buf + sizeof("do_domain_ids"), > + Domain_id_filter::DO_DOMAIN_IDS)) > + { > + sql_print_error("Failed to initialize master info do_domain_ids"); > + goto errwithmsg; > + } > + > + // IGNORE_DOMAIN_IDS > + if (0 == strncmp(buf, STRING_WITH_LEN("ignore_domain_ids=")) && > + mi->domain_id_filter.init_ids(buf + sizeof("ignore_domain_ids"), > + Domain_id_filter::IGNORE_DOMAIN_IDS)) > + { > + sql_print_error("Failed to initialize master info " > + "ignore_domain_ids"); > + goto errwithmsg; > + } So here you use the same style as adjacent code (that I wrote). However, I have since learned that the style (0 == strncmp(...)) is not used in the server. So please change this, as well as my existing code, to use either strncmp(...) == 0 or !strncmp(), as you prefer. Another thing from my old code is the sizeof("do_domain_ids"), which highly confused me for a bit before I realised that while the '=' is not included in the sizeof(), the terminating '\0' _is_ included. So since even I myself is confused by this code, better clarify it ;-) Perhaps it would be clearer if changed to (sizeof("do_domain_ids=")-1) (both in my old code and in your new code). (Normally it's good to follow the style of existing code as you did, I just noticed this in the review and realised that my own "existing code" would benefit from these small improvements). > @@ -634,6 +653,16 @@ > } > } > > + char* do_domain_ids_buf= > + mi->domain_id_filter.as_string(Domain_id_filter::DO_DOMAIN_IDS); > + if (do_domain_ids_buf == NULL) > + DBUG_RETURN(1); > + > + char* ignore_domain_ids_buf= > + mi->domain_id_filter.as_string(Domain_id_filter::IGNORE_DOMAIN_IDS); > + if (ignore_domain_ids_buf == NULL) > + DBUG_RETURN(1); You have a memory leak here, do_domain_ids_buf will not be freed if ignore_domain_ids_buf fails due to out-of-memory. > @@ -1348,4 +1382,218 @@ > DBUG_RETURN(result); > } > > +/* ctor */ > +Domain_id_filter::Domain_id_filter() : m_filter(false) > +{ > + for (int i= DO_DOMAIN_IDS; i <= IGNORE_DOMAIN_IDS; i ++) > + { > + my_init_dynamic_array(&m_domain_ids[i], sizeof(uint32), 16, 16, MYF(0)); > + } > +} > + > +/* dtor */ > +Domain_id_filter::~Domain_id_filter() I don't think comments like /* ctor */ and /* dtor */ are useful, better just omit them. > +/** > + Update the domain id list of the given type with elements from the new list. > + > + @param new_ids [IN] new list of ids > + @param type [IN] domain id list type to update > + @param changed [IN] has the domain id list changed in the > + last CHANGE MASTER command? > + > + @retval void > +*/ > +void Domain_id_filter::update_ids(DYNAMIC_ARRAY *new_ids, enum_list_type type, > + bool changed) > +{ > + DYNAMIC_ARRAY *ids= &m_domain_ids[type]; > + > + if (changed == true) > + reset_dynamic(ids); > + > + /* bsearch requires an ordered list. */ > + sort_dynamic(new_ids, change_master_domain_id_cmp); > + > + for (uint i= 0; i < new_ids->elements; i++) > + { > + uint32 domain_id; > + get_dynamic(new_ids, (void *) &domain_id, i); > + > + if (bsearch((const uint32 *) &domain_id, ids->buffer, ids->elements, > + sizeof(uint32), change_master_domain_id_cmp) == NULL) > + { > + insert_dynamic(ids, (uint32*) &domain_id); > + } > + } > + return; > +} Ok, so I understand that this code is modelled after the existing code for IGNORED_SERVER_IDS. But I found it rather confusing, because it looks like there is a bug: if changed==false, then the code does not ensure that the resulting array will still be sorted. I guess the bug is not possible to trigger, because the list of stuff to add will only be non-empty if changed==true? But that is rather confusing. Please simplify the logic; maybe you only need to call this function when changed==true, so the parameter can be omitted and the array always reset? Also, consider if you can rewrite the existing code for IGNORE_SERVER_IDS so that the same common code can be used as for IGNORE_DOMAIN_IDS and DO_DOMAIN_IDS, that would simplify the code and reduce risk of bugs or inconsistencies. > +/** > + Initialize the given domain id list with the numbers from the specified > + buffer. > + > + @param buf [IN] buffer > + @param type [IN] domain id list type > + > + @retval false Success > + true Error > +*/ > +bool Domain_id_filter::init_ids(char *buf, enum_list_type type) > +{ > + char *token, *last; > + uint num_items; > + DYNAMIC_ARRAY *ids= &m_domain_ids[type]; > + > + token= strtok_r(buf, " ", &last); > + if (token == NULL) > + { > + return true; /* error */ > + } > + > + num_items= atoi(token); > + for (uint i= 0; i < num_items; i++) > + { > + token= strtok_r(NULL, " ", &last); > + if (token == NULL) > + { > + return true; /* error */ > + } > + else > + { > + uint32 val= atoi(token); > + insert_dynamic(ids, (uchar *) &val); > + } > + } > + return false; > +} Again, did you consider using the same code for this and for IGNORE_SERVER_IDS? (I think you did, because you added a comment to init_dynarray_intvar_from_file ;-). Maybe consider it again; it is fine to rewrite/remove the existing code for IGNORE_SERVER_IDS and replace it with your new code. Again, it would reduce risk for bugs or inconsistencies. > + /* getter */ > + bool get_filter() { return m_filter; } > + > + /* setter */ > + void set_filter(uint32 domain_id); These are not really getter and setters, I think, at least I find those names confusing. Maybe something like do_filter(uint32 domain_id) and is_group_filtered() ? And expand the comments, so that they explain that these functions respectively check a GTID for needing to be filtered, and return whether the current group is being filtered. Also, personally I dislike setter/getter, and would prefer just to access the m_filter as a public member. But I suppose that's style and up to you. > + /* > + Initialize the given domain id list with the numbers from the specified > + buffer. > + > + @param buf [IN] buffer > + @param type [IN] domain id list type > + > + @retval false Success > + true Error > + */ > + bool init_ids(char *buf, enum_list_type type); Clarify in the comment that the list is initialized from a string of space-separated numbers, the first of which is the number of following entries. > @@ -182,6 +279,7 @@ > bool flush_relay_log_cache, > bool need_lock_relay_log); > int change_master_server_id_cmp(ulong *id1, ulong *id2); > +int change_master_domain_id_cmp(uint32 *id1, uint32 *id2); I think change_master_domain_id_cmp() is only used in rpl_mi.cc, so should be static there, and not refered here in rpl_mi.h > === modified file 'sql/slave.cc' > --- a/sql/slave.cc 2014-08-12 03:55:41 +0000 > +++ b/sql/slave.cc 2014-09-12 18:16:22 +0000 > @@ -1259,6 +1259,7 @@ > going to ignore (to not log them in the relay log). > Items being read are supposed to be decimal output of values of a > type shorter or equal of @c long and separated by the single space. > + It also used to restore DO_DOMAIN_IDS & IGNORE_DOMAIN_IDS lists. This comment suggests you wanted to share code, then later changed your mind. If you end up keeping the code separate, you still need to delete this comment. (And there is a typo: "It also" -> "It is also", but if you remove it anyway...). > @@ -5601,6 +5610,10 @@ > mi->last_queued_gtid= event_gtid; > mi->last_queued_gtid_standalone= > (gtid_flag & Gtid_log_event::FL_STANDALONE) != 0; > + > + /* Should filter all the subsequent events in the current GTID group? */ > + mi->domain_id_filter.set_filter(event_gtid.domain_id); So, here we will set m_filter to true, if we want to skip the following group. But I don't see any code that will set it to false at the end of the group? That seems a problem, as we could then end up wrongly skipping all kinds of other events, such as Gtid_list, Incident, ... Also note that it is possible for a GTID group to not be terminated by COMMIT/XID. This can happen at least if the master crashes in the middle of writing a group; the slave will receive only the first part of it, followed by a master restart Format_description event. That case needs to be handled, and you should add a test case for it. Also it is possible for eg. a master downgrade to introduce following event groups that do not start with GTID - seems these would also be wrongly skipped? So we also need a test case for this... Hope this helps, - Kristian.

2 8

[Maria-developers] The variable "table_definition_cache" is autosized by default on MySQL 5.6.8+ but not on MariaDB 10
by Jean Weisbuch 20 Nov '14

20 Nov '14

The value of the variable "table_definition_cache" is by default autosized (based on the table_open_cache value) on MySQL >=5.6.8 while it is still static (400) on the latest MariaDB 10.0.14 and 10.1.?, is that by choice or this feature hasn't been backported and should be? From the MySQL doc http://dev.mysql.com/doc/refman/5.6/en/server-system-variables.html#sysvar_… : *Permitted Values (>= 5.6.8)* *Type* |numeric| *Default* |-1 (autosized)| *Min Value* |400| *Max Value* |524288| The default value is based on the following formula, capped to a limit of 2000: 400 + (table_open_cache / 2) Before MySQL 5.6.8, the default is 400.

1 0

Re: [Maria-developers] [Commits] Rev 4367: MDEV-7026: Race in InnoDB/XtraDB mutex implementation can stall or hang the server. in http://bazaar.launchpad.net/~maria-captains/maria/5.5
by Sergey Vojtovich 20 Nov '14

20 Nov '14

Hi Kristian, thanks for taking over this bug! I believe this is the patch that was pushed into 5.5. Unfortunately I think there is still one potential for a hang left (details below). :( Since InnoDB and XtraDB changes are identical I'll comment only XtraDB changes. So you removed os_isync and implemented os_mb instead. That's good! You also removed a bunch of os_wmb/os_rmb calls. That's good too. And you kept RELEASE barrier for mutex_exit() on PPC64. Very good! I will also comment things that I don't really like, but they're just for your information (no action required). On Wed, Nov 19, 2014 at 01:56:46PM +0100, knielsen(a)knielsen-hq.org wrote: > At http://bazaar.launchpad.net/~maria-captains/maria/5.5 > > ------------------------------------------------------------ > revno: 4367 > revision-id: knielsen(a)knielsen-hq.org-20141119125646-njq02h7irmu9s71b > parent: sergii(a)pisem.net-20141118231952-fk3o69075dn3ravv > committer: Kristian Nielsen <knielsen(a)knielsen-hq.org> > branch nick: mariadb-5.5 > timestamp: Wed 2014-11-19 13:56:46 +0100 > message: > MDEV-7026: Race in InnoDB/XtraDB mutex implementation can stall or hang the server. > > The bug was that full memory barrier was missing in the code that ensures that > a waiter on an InnoDB mutex will not go to sleep unless it is guaranteed to be > woken up again by another thread currently holding the mutex. This made > possible a race where a thread could get stuck waiting for a mutex that is in > fact no longer locked. If that thread was also holding other critical locks, > this could stall the entire server. There is an error monitor thread than can > break the stall, it runs about once per second. But if the error monitor > thread itself got stuck or was not running, then the entire server could hang > infinitely. > > This was introduced on i386/amd64 platforms in 5.5.40 and 10.0.13 by an > incorrect patch that tried to fix the similar problem for PowerPC. JFYI: Strictly speaking that "incorrect patch" attempted to fix different problem (read carefully): it added RELEASE barrier between "stores inside mutex" and "lock_word= 0" while this patch adds full barrier between "lock_word= 0" and "if (waiters)". > > This commit reverts the incorrect PowerPC patch, and instead implements a fix > for PowerPC that does not change i386/amd64 behaviour, making PowerPC work > similarly to i386/amd64. ...skip... > === modified file 'storage/xtradb/include/os0sync.h' > --- a/storage/xtradb/include/os0sync.h 2014-09-08 15:10:48 +0000 > +++ b/storage/xtradb/include/os0sync.h 2014-11-19 12:56:46 +0000 > @@ -314,11 +331,28 @@ amount of increment. */ > /**********************************************************//** > Returns the old value of *ptr, atomically sets *ptr to new_val */ > > -# define os_atomic_test_and_set_byte(ptr, new_val) \ > +#ifdef __powerpc__ > +/* > + os_atomic_test_and_set_byte_release() should imply a release barrier before > + setting, and a full barrier after. But __sync_lock_test_and_set() is only > + documented as an aquire barrier. So on PowerPC we need to add the full > + barrier explicitly. */ > +# define os_atomic_test_and_set_byte_release(ptr, new_val) \ > + do { __sync_lock_release(ptr); \ > + __sync_synchronize(); } while (0) > +#else > +/* > + On x86, __sync_lock_test_and_set() happens to be full barrier, due to > + LOCK prefix. > +*/ > +# define os_atomic_test_and_set_byte_release(ptr, new_val) \ > + __sync_lock_test_and_set(ptr, (byte) new_val) > +#endif JFYI: "os_atomic_test_and_set_byte_release" sounds odd to me. Look at gcc manual: https://gcc.gnu.org/onlinedocs/gcc-4.1.2/gcc/Atomic-Builtins.html ... This builtin is not a full barrier, but rather an acquire barrier. This means that references after the builtin cannot move to (or be speculated to) before the builtin, but previous memory stores may not be globally visible yet, and previous memory loads may not yet be satisfied. ... You name RELEASE something that is acqually ACQUIRE. So why didn't you like "os_atomic_lock_release_byte" which is RELASE btw: ... This builtin is not a full barrier, but rather a release barrier. This means that all previous memory stores are globally visible, and all previous memory loads have been satisfied, but following memory reads are not prevented from being speculated to before the barrier. ... > +/* > + os_atomic_test_and_set_byte_acquire() is a full memory barrier on x86. But > + in general, just an aquire barrier should be sufficient. */ > +# define os_atomic_test_and_set_byte_acquire(ptr, new_val) \ > __sync_lock_test_and_set(ptr, (byte) new_val) !!! The problem is here!!! Strictly speaking not here, but in mutex_spin_wait(): ... mutex_set_waiters(mutex, 1); /* Try to reserve still a few times */ for (i = 0; i < 4; i++) { if (mutex_test_and_set(mutex) == 0) { ... We'll get stuck if "waiters= 1" will become visible after "if (mutex_test_and_set(mutex))". I don't see anything preventing this reorder. ...skip... > === modified file 'storage/xtradb/include/sync0sync.ic' > --- a/storage/xtradb/include/sync0sync.ic 2014-08-29 12:02:46 +0000 > +++ b/storage/xtradb/include/sync0sync.ic 2014-11-19 12:56:46 +0000 > @@ -80,11 +80,11 @@ mutex_test_and_set( > mutex_t* mutex) /*!< in: mutex */ > { > #if defined(HAVE_ATOMIC_BUILTINS) > - return(os_atomic_test_and_set_byte(&mutex->lock_word, 1)); > + return(os_atomic_test_and_set_byte_acquire(&mutex->lock_word, 1)); > #else > ibool ret; > > - ret = os_fast_mutex_trylock(&(mutex->os_fast_mutex)); > + ret = os_fast_mutex_trylock_full_barrier(&(mutex->os_fast_mutex)); > > if (ret == 0) { > /* We check that os_fast_mutex_trylock does not leak JFYI: This is probably the most funny thing. Here we're in the "#else" branch of "#if defined(HAVE_ATOMIC_BUILTINS)". E.g. if we compile with gcc and enter this branch it means: sync builtins are not avialble (gcc atomic builtins are very likely not available too). Now you say "os_fast_mutex_trylock_full_barrier" which does os_mb. But since neither sync bultins nor atomic builtins are available, os_mb is basically no-op. This may be different on other platforms, but to my knowledge memory barriers and atomic operations normally come together. Another thing: did you try compiling with this branch enabled? IIRC it didn't compile even berfore we started fixing it. ...skip... Regards, Sergey

2 2

[Maria-developers] constant propagation & LIKE
by Alexander Barkov 20 Nov '14

20 Nov '14

Hello Sergei, The attached patch fixes MDEV-7149 Constant propagation erroneously applied for LIKE All tests pass. But I'm not 100% sure about the patch. Is eq_cmp_result() proper place to fix this? If it's the right place, I can test the collation properties and return: - COND_TRUE for simple collations with one-character-to-one-weight rules, - COND_OK for the complex collations with tricky character to weight mapping. Thanks.

1 0

Re: [Maria-developers] [Commits] Rev 4358: MDEV-7113 difference between check_vcol_func_processor and check_partition_func_processor in lp:~maria-captains/maria/5.5
by Sergey Petrunia 20 Nov '14

20 Nov '14

Hi Sergei, * what about persistent virtual columns? Do we still support environment-dependent virtual functions for those? * the behavior of current solution is not deterministic: thread1> select @@time_zone; +-------------+ | @@time_zone | +-------------+ | SYSTEM | +-------------+ 1 row in set (0.00 sec) thread1> create table t2 ( a varchar(32), b int, v varchar(100) as (from_unixtime(b))); Query OK, 0 rows affected (0.28 sec) thread1> insert into t2 (a,b) values ('local-tz-23-11', unix_timestamp()); Query OK, 1 row affected (0.04 sec) thread1> select * from t2; +----------------+------------+---------------------+ | a | b | v | +----------------+------------+---------------------+ | local-tz-23-11 | 1416427891 | 2014-11-19 23:11:31 | +----------------+------------+---------------------+ 1 row in set (0.00 sec) thread2> set time_zone='Europe/Helsinki'; Query OK, 0 rows affected (0.00 sec) thread2> select * from t2; +----------------+------------+---------------------+ | a | b | v | +----------------+------------+---------------------+ | local-tz-23-11 | 1416427891 | 2014-11-19 23:11:31 | +----------------+------------+---------------------+ 1 row in set (0.00 sec) thread2> flush tables; Query OK, 0 rows affected (0.00 sec) thread2> select * from t2; +----------------+------------+---------------------+ | a | b | v | +----------------+------------+---------------------+ | local-tz-23-11 | 1416427891 | 2014-11-19 22:11:31 | +----------------+------------+---------------------+ 1 row in set (0.00 sec) Is this really ok? Maybe, it makes sense to use some some "default" (e.g. global variables) environment on the non-deterministic virtual column functions? Or, make these functions always fetch the environment from the current thread? On Mon, Nov 17, 2014 at 09:43:46AM +0100, Sergei Golubchik wrote: > At lp:~maria-captains/maria/5.5 > > ------------------------------------------------------------ > revno: 4358 > revision-id: sergii(a)pisem.net-20141115155753-rx5twf6utu1isczh > parent: sergii(a)pisem.net-20141113173729-hirl67og6at3xbni > fixes bugs: https://mariadb.atlassian.net/browse/MDEV-6789 https://mariadb.atlassian.net/browse/MDEV-7113 > committer: Sergei Golubchik <sergii(a)pisem.net> > branch nick: 5.5 > timestamp: Sat 2014-11-15 16:57:53 +0100 > message: > MDEV-7113 difference between check_vcol_func_processor and check_partition_func_processor > MDEV-6789 segfault in Item_func_from_unixtime::get_date on updating table with virtual columns > > * prohibit VALUES in partitioning expression > * prohibit user and system variables in virtual column expressions > * fix Item_func_date_format to cache locale (for %M/%W to return the same as MONTHNAME/DAYNAME) > * fix Item_func_from_unixtime to cache time_zone directly, not THD (and not to crash) > * added tests for other incorrectly allowed (in vcols) functions to see that they don't crash ... BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

2 2

Re: [Maria-developers] [Commits] Rev 4346: MDEV-7026 - Occasional hang during startup on Power8 in lp:maria/5.5
by Kristian Nielsen 19 Nov '14

19 Nov '14

Sergey Vojtovich <svoj(a)mariadb.org> writes: > revno: 4346 > revision-id: svoj(a)mariadb.org-20141118080742-sdf93kgsgxulenvb > parent: knielsen(a)knielsen-hq.org-20141112101013-t63ayylsk08ncgs3 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 5.5 > timestamp: Tue 2014-11-18 12:07:42 +0400 > message: > MDEV-7026 - Occasional hang during startup on Power8 Right, so you requested second review ASAP, I'm trying to understand what's going on here. If my understanding is correct, this patch goes on top of current 5.5.40, not on top of the patch I reviewed previously. So there are two cases, depending on whether INNODB_RW_LOCKS_USE_ATOMICS is set or not. It is my understanding that this is "usually" set, eg. on our common x86 and powerpc linux binaries. The critical thing for the patch at hand is the memory barrier between mutex_set_waiters() and mutex_test_and_set() in the waiter, and mutex_reset_lock_word() and mutex_get_waiters() in the thread releasing the lock. A full memory barrier is needed in the waiter, to be sure that mutex_set_waiters() becomes visible to the releasing thread before mutex_test_and_set() runs. And a full barrier is needed in the releasing thread, to be sure that mutex_reset_lock_word() becomes visible before mutex_get_waiters() runs. So let's see how this is ensured. First the case of x86 with INNODB_RW_LOCKS_USE_ATOMICS set, HAVE_ATOMIC_BUILTINS, pre-5.5.40: - mutex_set_waiters() uses os_compare_and_swap_ulint() which is __sync_bool_compare_and_swap() which is full barrier. - mutex_reset_lock_word() uses os_atomic_test_and_set_byte() which is __sync_lock_test_and_set(). This is documented only as a release barrier, but is in fact a full barrier on x86 with current GCC. So in this case full barrier is there as needed. If HAVE_ATOMIC_BUILTINS is not set, then mutex_reset_lock_word() uses os_fast_mutex_unlock() which is pthread_mutex_unlock(), which is also full barrier on x86. If INNODB_RW_LOCKS_USE_ATOMICS is not set, then mutex_set_waiters() has no barriers. But then mutex_test_and_set() instead has a full barrier on x86. Either in os_atomic_test_and_set_byte() which is __sync_lock_test_and_set(), if HAVE_ATOMIC_BUILTINS is set, or in os_fast_mutex_trylock() otherwise. Both of these are full barriers on x86. So this is why things work ok on x86, apparently. Very bad code that seriously needs cleanup and proper documentation of memory barrier semantics. But it is what we have to work with in 5.5. So to make things work on PowerPC without risk of breaking x86, we could do this: - Make sure os_atomic_test_and_set_byte() is full barrier on PowerPC: #ifdef __powerpc__ /* os_atomic_test_and_set_byte() should imply a full barrier. But __sync_lock_test_and_set() is only documented as an aquire barrier. So on PowerPC we need to add the full barrier explicitly. */ # define os_atomic_test_and_set_byte(ptr, new_val) \ do { __sync_lock_test_and_set(ptr, (byte) new_val); \ __atomic_thread_fence(__ATOMIC_SEQ_CST); } while(0) #else /* On x86, __sync_lock_test_and_set() happens to be full barrier, due to LOCK prefix. */ # define os_atomic_test_and_set_byte(ptr, new_val) \ __sync_lock_test_and_set(ptr, (byte) new_val) #endif - Make sure that mutex_reset_lock_word() is a full barrier when HAVE_ATOMIC_BUILTINS is not set, by adding a memory barrier: os_fast_mutex_unlock(&(mutex->os_fast_mutex)); #ifdef __powerpc__ /* On PowerPC, pthread_mutex does not imply full barrier, so we need to add it explicitly. */ __atomic_thread_fence(__ATOMIC_SEQ_CST); #endif - For the case of neither HAVE_ATOMIC_BUILTINS nor INNODB_RW_LOCKS_USE_ATOMICS set, similarly add a full barrier in mutex_test_and_set() for PowerPC: #ifdef __powerpc__ /* On PowerPC, pthread_mutex does not imply full barrier, so we need to add it explicitly. */ __atomic_thread_fence(__ATOMIC_SEQ_CST); #endif ret = os_fast_mutex_trylock(&(mutex->os_fast_mutex)); Then in 10.1. if someone is brave enough, we can clean this up properly. Clearly define what the memory barrier semantics is, and implement it using the correct primitives that work as needed on all platforms. So now that I understand (hopefully...) how the code used to work, and how it can be fixed for PowerPC without changing existing code, let me look at the new patch ... > === modified file 'storage/innobase/include/os0sync.h' > --- a/storage/innobase/include/os0sync.h 2014-09-08 15:10:48 +0000 > +++ b/storage/innobase/include/os0sync.h 2014-11-18 08:07:42 +0000 > @@ -310,8 +310,16 @@ Returns the old value of *ptr, atomicall > # define os_atomic_test_and_set_byte(ptr, new_val) \ > __sync_lock_test_and_set(ptr, (byte) new_val) > > +#ifdef __powerpc__ > # define os_atomic_lock_release_byte(ptr) \ > __sync_lock_release(ptr) > +#else > +/* In theory __sync_lock_release should be used to release the lock. > +Unfortunately, it does not work properly alone. The workaround is > +that more conservative __sync_lock_test_and_set is used instead. */ > +# define os_atomic_lock_release_byte(ptr) \ > + __sync_lock_test_and_set(ptr, 0) > +#endif This is very confusing. You define a method with full barrier semantics, but call it something suggesting release semantics. Why not revert the original change so that mutex_reset_lock_word() uses os_atomic_test_and_set_byte()? And then fix the implementation of os_atomic_test_and_set_byte() to be full barrier on PowerPC, as I suggested above? The term "atomic" generally implies full barrier in the MySQL source code... > #elif defined(HAVE_IB_SOLARIS_ATOMICS) > > @@ -428,7 +436,7 @@ clobbered */ > # define os_rmb __atomic_thread_fence(__ATOMIC_ACQUIRE) > # define os_wmb __atomic_thread_fence(__ATOMIC_RELEASE) > #ifdef __powerpc__ > -# define os_isync __asm __volatile ("isync":::"memory") > +# define os_isync __atomic_thread_fence(__ATOMIC_SEQ_CST) > #else > #define os_isync do { } while(0) > #endif As Jonas suggested, don't call a full memory barrier "isync". And do not introduce something that is a barrier on one platform but a no-operation on another platform. But ... > === modified file 'storage/innobase/sync/sync0sync.c' > --- a/storage/innobase/sync/sync0sync.c 2014-11-03 13:43:44 +0000 > +++ b/storage/innobase/sync/sync0sync.c 2014-11-18 08:07:42 +0000 > @@ -474,10 +474,9 @@ mutex_set_waiters( > > ptr = &(mutex->waiters); > > - os_wmb; > - > *ptr = n; /* Here we assume that the write of a single > word in memory is atomic */ > + os_isync; > } With this, the semantics of how things work becomes even more complicated. Now on PowerPC the barrier is in one function, but on x86 the barrier is in another function. Why not instead do as I suggested above? Put in full barriers for PowerPC in the same places that they are in x86? ---- Maybe I should try to say if I think the patch fixes the actual bug, in case you don't want to go with my suggestions and instead keep your approach? Since it goes back to using __sync_lock_test_and_set() on x86, and since it has a full barrier in mutex_exit_func() on PowerPC, I think perhaps it does. But it is really hard to understand exactly what is going on, as you can see above I had to do a lot of reasoning to understand even why the old code worked... My advice is still to revert the original incorrect PowerPC patch and do something that is at least consistent with the existing code... (And then feel free to do a proper cleanup in 10.1). Hope this helps, - Kristian.

2 2

Re: [Maria-developers] [Commits] Rev 4346: MDEV-7026 - Occasional hang during startup on Power8 in lp:maria/5.5
by Jonas Oreland 18 Nov '14

18 Nov '14

> > Since isync is not a memory barrier and lwsync doesn't guarantee > StoreLoad > order, the only option we have is full memory barrier. > === modified file 'storage/innobase/include/os0sync.h' > --- a/storage/innobase/include/os0sync.h 2014-09-08 15:10:48 +0000 > +++ b/storage/innobase/include/os0sync.h 2014-11-18 08:07:42 +0000 > > #ifdef __powerpc__ > -# define os_isync __asm __volatile ("isync":::"memory") > +# define os_isync __atomic_thread_fence(__ATOMIC_SEQ_CST) > you have to rename to the #define too (e.g to os_mb). having a os_isync which is infact a mb is more obfuscation than anyone should have to put up with. /Jonas

2 1

Re: [Maria-developers] [Commits] Rev 4346: MDEV-7026 - Occasional hang during startup on Power8 in lp:maria/5.5
by Kristian Nielsen 17 Nov '14

17 Nov '14

Sergey Vojtovich <svoj(a)mariadb.org> writes: > revno: 4346 > revision-id: svoj(a)mariadb.org-20141112114907-thnro6e3kg2ofdsw > committer: Sergey Vojtovich <svoj(a)mariadb.org> > timestamp: Wed 2014-11-12 15:49:07 +0400 > message: > MDEV-7026 - Occasional hang during startup on Power8 > === modified file 'storage/xtradb/sync/sync0sync.c' > --- a/storage/xtradb/sync/sync0sync.c 2014-11-03 13:43:44 +0000 > +++ b/storage/xtradb/sync/sync0sync.c 2014-11-12 11:49:07 +0000 > @@ -467,26 +467,15 @@ mutex_set_waiters( > mutex_t* mutex, /*!< in: mutex */ > ulint n) /*!< in: value to set */ > { > -#ifdef INNODB_RW_LOCKS_USE_ATOMICS > - ut_ad(mutex); > - > - if (n) { > - os_compare_and_swap_ulint(&mutex->waiters, 0, 1); > - } else { > - os_compare_and_swap_ulint(&mutex->waiters, 1, 0); > - } > -#else > volatile ulint* ptr; /* declared volatile to ensure that > the value is stored to memory */ > ut_ad(mutex); > > ptr = &(mutex->waiters); > > - os_wmb; > - > *ptr = n; /* Here we assume that the write of a single > word in memory is atomic */ > -#endif > + os_sync; > } This code is still substantially different from what it was in 5.5.39 and 10.0.12, right? I think we need to instead revert the code to what it was before introducing the problem. The InnoDB mutex code clearly does not work by careful design - mutex_reset_lock_word() needs a full barrier but uses what is documented by GCC as only an acquire barrier. Rather, it works by having been tweaked over the years to just happen to work (eg. GCC in fact uses a full barrier on x86). So changing this to be "correct" is very dangerous in a stable release. It is not possible to ensure correctness by testing, either. But it is not necessary to change it. Instead, leave the code as it was before, and use #ifdef __powerpc__ to ensure that it will work also on PowerPC. For example, something like this (just a sketch, I didn't check exactly how the code should look): #ifdef __powerpc__ /* InnoDB mutex implementation needs full memory barrier for this operation */ # define os_atomic_test_and_set_byte(ptr, new_val) \ do { sync(); __sync_lock_test_and_set(ptr, (byte) new_val); } while (0) #else # define os_atomic_test_and_set_byte(ptr, new_val) \ __sync_lock_test_and_set(ptr, (byte) new_val) #endif Then we leave stable releases 5.5 and 10.0 unchanged, reducing the risk of introducing another serious breakage. And we can still fix PowerPC to work correctly. And then perhaps in 10.1 we can do refactoring of the use of memory barriers in InnoDB, if we really want. (Just in case it's not obvious, yes, I _really_ have the InnoDB mutex code). Remember, the 5.5 and 10.0 minor releases appear as security updates in distros, so it is very important to minimise the risk of introducing serious regressions. > Since isync is not a memory barrier and lwsync doesn't guarantee StoreLoad > order, the only option we have is full memory barrier. It's not a problem to have a full barrier here, I think. Because at this point, the waiter has already gone through a lot of spin loops, and now it is going to take a global mutex on the sync wait array, scan the array, and go to sleep with a context switch. One less full barrier is not going to help a lot. To improve this contended case of the InnoDB mutexes, more radical changes are needed. > InnoDB/XtraDB was getting stuck during server startup. In particular (but not > limited to) while creating doublewrite buffer. The stuck during startup is kind of the minor part of the bug. The major part is stalling the server during operation. (The hang during startup was just how the problem was detected in Buildbot). Better clarify in the commit message that this fixes a bug that could cause the entire server to stall (or even hang completely if error monitor thread itself got stuck). This will help people who experienced the problem and look at the changelog. > It happens because mutex_exit() didn't wake up waiters of affected mutex. This > is kind of acceptable because error monitor thread is supposed to wake up such > waiters. But since error monitor thread hasn't been started that early, nobody > did inform waiting threads that mutex was released. I do not think the error monitor is "supposed to wake up such waiters". In fact, it seems clear that the error monitor wakeup is _not_ used, at least not in commonly used binaries. Because it just never worked - it itself needs to take mutexes, which will prevent it from doing the wakeup. So if it was needed, someone would have noticed this problem. The symptom is _very_ hard to miss, as the whole server gets stuck hard, unable to do anything. It really is not acceptable to have a mutex implementation that can miss wakeups. This can stall the entire server. Even if the error monitor resolves the stall after 1 second, that is still unacceptable for many applications. Hope this helps, - Kristian.

2 3

[Maria-developers] Review: [MDEV-7062] parser refactoring: don't store field properties in LEX
by Alexander Barkov 16 Nov '14

16 Nov '14

Hi Sergei, > Sergei Golubchik edited comment on MDEV-7062 at 11/10/14 11:40 AM: > ------------------------------------------------------------------- > > http://lists.askmonty.org/pipermail/commits/2014-November/006908.html > > {noformat} > From: serg(a)mariadb.org > To: commits(a)mariadb.org > Subject: [Commits] 2c904b9: parser cleanup: don't store field properties > in LEX, use Create_field directly > Date: Mon, 10 Nov 2014 10:26:48 +0100 > {noformat} > Please find my comments below: > > [Commits] 2c904b9: parser cleanup: don't store field properties in LEX, use Create_field directly > serg at mariadb.org serg at mariadb.org > Mon Nov 10 11:26:48 EET 2014 > > Previous message: [Commits] Rev 4344: MDEV-6179: dynamic columns functions/cast()/convert() doesn't play nice with CREATE/ALTER TABLE in lp:~maria-captains/maria/5.5 > Next message: [Commits] Rev 4345: MDEV-6862 "#error <my_config.h>" and third-party libraries in lp:~maria-captains/maria/5.5 > Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] > > revision-id: 2c904b937b5ef7fbec60ebb9cb514a501c5bd8e4 > parent(s): f59588b9112965849986c45fcd150d9abed74063 > committer: Sergei Golubchik > branch nick: maria > timestamp: 2014-11-09 22:33:17 +0100 > message: > > parser cleanup: don't store field properties in LEX, use Create_field directly > > length/dec/charset are still in LEX, because they're also used > for CAST and dynamic columns. > > also > 1. fix "MDEV-7041 COLLATION(CAST('a' AS CHAR BINARY)) returns a wrong result" > 2. allow BINARY modifier in stored function RETURN clause > 3. allow "COLLATION without CHARSET" in SP/SF (parameters, RETURN, DECLARE) > 4. print correct variable name in error messages for stored routine parameters A very nice clean up. I only found it a little bit not easy to track that all Lex and Create_field attributes get initialized and copied to each other properly. I suggest to go a little bit further and get rid from Lex->dec, Lex->length and Lex->charset at all, and write the attributes directly to the destination as soon as they get parsed. See a proposed addon patch attached. > diff --git a/mysql-test/r/cast.result b/mysql-test/r/cast.result > index c81af13..8ad4568 100644 > --- a/mysql-test/r/cast.result > +++ b/mysql-test/r/cast.result > @@ -796,3 +796,32 @@ DATE("foo") > NULL > Warnings: > Warning 1292 Incorrect datetime value: 'foo' > +create table t1 (a int, b char(5) as (cast("a" as char(10) binary) + a) ); > +show create table t1; > +Table Create Table > +t1 CREATE TABLE `t1` ( > + `a` int(11) DEFAULT NULL, > + `b` char(5) AS (cast("a" as char(10) binary) + a) VIRTUAL > +) ENGINE=MyISAM DEFAULT CHARSET=latin1 > +drop table t1; > +select collation(cast("a" as char(10) binary)); > +collation(cast("a" as char(10) binary)) > +latin1_bin > +select collation(cast("a" as char(10) charset utf8 binary)); > +collation(cast("a" as char(10) charset utf8 binary)) > +utf8_bin > +select collation(cast("a" as char(10) ascii binary)); > +collation(cast("a" as char(10) ascii binary)) > +latin1_bin > +select collation(cast("a" as char(10) unicode binary)); > +collation(cast("a" as char(10) unicode binary)) > +ucs2_bin UNICODE is an alias for UCS2, which is not always compiled. Please move ucs2 related tests to ctype_ucs.test. <skip> > diff --git a/mysql-test/t/cast.test b/mysql-test/t/cast.test > index b6c37ca..aa43ee6 100644 > --- a/mysql-test/t/cast.test > +++ b/mysql-test/t/cast.test > @@ -456,3 +456,21 @@ SELECT CAST(TIME('10:20:30') AS DATE) + INTERVAL 1 DAY; > SET SQL_MODE=ALLOW_INVALID_DATES; > SELECT DATE("foo"); > > +# > +# CAST and field definition using same fields in LEX > +# > +create table t1 (a int, b char(5) as (cast("a" as char(10) binary) + a) ); > +show create table t1; > +drop table t1; > + > +# > +# CAST (... BINARY) > +# > +select collation(cast("a" as char(10) binary)); > +select collation(cast("a" as char(10) charset utf8 binary)); > +select collation(cast("a" as char(10) ascii binary)); > +select collation(cast("a" as char(10) unicode binary)); > +select collation(cast("a" as char(10) binary charset utf8)); > +select collation(cast("a" as char(10) binary ascii)); > +select collation(cast("a" as char(10) binary unicode)); More UCS2. You removed this code in field.cc: > - if (fld_length != NULL) > - { > - errno= 0; > - length= strtoul(fld_length, NULL, 10); > - if ((errno != 0) || (length > MAX_FIELD_BLOBLENGTH)) > - { > - my_error(ER_TOO_BIG_DISPLAYWIDTH, MYF(0), fld_name, MAX_FIELD_BLOBLENGTH); > - DBUG_RETURN(TRUE); > - } > - > - if (length == 0) > - fld_length= NULL; /* purecov: inspected */ > - } and added this in sql_yacc.yy: > + if (length) > + { > + int err; > + last_field->length= (ulong)my_strtoll10(length, NULL, &err); > + if (err) > + last_field->length= ~0UL; // safety > + } > + else > + last_field->length= 0; I have a feeling that on a 32bit platforms it will stop sending errors on attempt to create a blob larger that 4Gb. last_field->length is ulong, which is 32bit is a 32 bit platform, and it gets assigned to a 64 bit value.

2 2

[Maria-developers] Please review: sql_yacc.yy and HA_CREATE_INFO refactoring (pre-requisite for MDEV-5359 CREATE OR REPLACE, CREATE IF NOT EXISTS, DROP IF EXISTS)
by Alexander Barkov 14 Nov '14

14 Nov '14

Hi Sergei, The patch is attached. Thanks.

1 1

Re: [Maria-developers] [JIRA] (MDEV-7076) Build fails around libgroonga on some platforms in buildbot
by Alexander Barkov 12 Nov '14

12 Nov '14

Hello Kentoku, Mroonga fails to compile on some platforms. Can you please have a look? Thanks. On 11/12/2014 04:39 PM, Alexander Barkov (JIRA) wrote: > > [ https://mariadb.atlassian.net/browse/MDEV-7076?page=com.atlassian.jira.plug… ] > > Alexander Barkov updated MDEV-7076: > ----------------------------------- > Assignee: Kentoku (was: Alexander Barkov) > >> Build fails around libgroonga on some platforms in buildbot >> ----------------------------------------------------------- >> >> Key: MDEV-7076 >> URL: https://mariadb.atlassian.net/browse/MDEV-7076 >> Project: MariaDB Server >> Issue Type: Bug >> Components: OTHER >> Affects Versions: 10.0 >> Reporter: Elena Stepanova >> Assignee: Kentoku >> Priority: Critical >> Labels: buildbot, compile >> Fix For: 10.0.15 >> >> >> http://buildbot.askmonty.org/buildbot/builders/kvm-bintar-centos5-amd64/bui… >> http://buildbot.askmonty.org/buildbot/builders/kvm-deb-debian6-amd64/builds… >> http://buildbot.askmonty.org/buildbot/builders/kvm-deb-lucid-amd64/builds/4… >> It looks approximately like this, with some variations >> {noformat} >> standard input}: Assembler messages: >> {standard input}:28252: Error: open CFI at the end of file; missing .cfi_endproc directive >> gcc-4.4.real: Internal error: Killed (program cc1) >> Please submit a full bug report. >> See <file:///usr/share/doc/gcc-4.4/README.Bugs> for instructions. >> make[3]: *** [storage/mroonga/vendor/groonga/lib/CMakeFiles/libgroonga.dir/expr.c.o] Error 1 >> make[3]: *** Waiting for unfinished jobs.... >> {noformat} >> Given the list of platforms, I guess they all feature pretty old compilers, maybe that's the distinction. > > > > -- > This message was sent by Atlassian JIRA > (v6.4-OD-09-008#64005) >

1 0

[Maria-developers] Mroonga: wrapper.version_56_or_later_performance_schema is failing on BuildBot
by Alexander Barkov 12 Nov '14

12 Nov '14

Hello Kentoku, This test is failing in the latest builds on BB: mroonga/wrapper.version_56_or_later_performance_schema http://buildbot.askmonty.org/buildbot/builders/bld-dan-release/builds/1793/… Can you please check it? Thanks.

2 1

[Maria-developers] PLEASE REVIEWA: pre-requisite patch for the data type plugins, also fixing a number of optimizer bugs
by Alexander Barkov 12 Nov '14

12 Nov '14

Hello, While working on pluggable data types, I noticed a few problems related to optimizer: > MDEV-6950 Bad results with joins comparing DATE/DATETIME and INT/DECIMAL/DOUBLE/ENUM/VARCHAR columns > MDEV-6971 Bad results with joins comparing TIME and DOUBLE/DECIMAL columns > MDEV-6978 Bad results with joins comparing case insensitive VARCHAR/ENUM/SET expression to a _bin ENUM column > MDEV-6982 LEFT JOIN table elimination is not always used when it could > MDEV-6989 BINARY and COLLATE xxx_bin comparisions are not used for optimization in some cases > MDEV-6990 GROUP_MIN_MAX optimization is not applied in some cases when it could > MDEV-6991 GROUP_MIN_MAX optimization is erroneously applied in some cases In some cases wrong result sets are returned. So it would be nice to have this fixed in 10.0. I made these changes as a standalone patch. Please review. I also noticed some other optimizer bugs but not sure how to fix them properly: > MDEV-6986 Bad results with join comparing INT and VARCHAR columns > MDEV-6969 Bad results with joins comparing DOUBLE to BIGINT/DECIMAL columns > MDEV-6993 Bad results with join comparing DECIMAL and ENUM/SET columns Igor or SergeyP can possibly fix them on top of my changes. Also, I found some other problems (not related to optimizer): > MDEV-6973 XOR aggregates argument collations > MDEV-7005 NULLIF does not work as documented Not sure which version to fix them in. Please suggest. Thanks.

2 2

Re: [Maria-developers] MDEV-6885 - debs do not build on PPC64 in lp:maria/5.5
by Sergei Golubchik 11 Nov '14

11 Nov '14

Hi, Sergey! On Oct 17, Sergey Vojtovich wrote: > At lp:maria/5.5 > > ------------------------------------------------------------ > revno: 4327 > revision-id: svoj(a)mariadb.org-20141017112032-b0p2qvvlbn1xrfec > parent: svoj(a)mariadb.org-20141017110850-5rlt8fm3blxv4wtb > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 5.5 > timestamp: Fri 2014-10-17 15:20:32 +0400 > message: > MDEV-6885 - debs do not build on PPC64 > > Include ha_oqgraph.so into deb package only if it was able to build. Sorry, cannot help much. It looks ok, but I've had so many problems with our debian packaging - all these fixes looks right, and still need twenty iterations to actually be right. So, try it in buildbot, push in some tree that builds deb packages. If it'll work in all deb builders - please, push into 5.5. Regards, Sergei

3 3

Re: [Maria-developers] [Commits] MDEV-60 Support for Spatial Reference systems for the GIS data.
by Sergei Golubchik 11 Nov '14

11 Nov '14

Hi, Alexey! First, please, please, *do not* send patches from Thunderbird. It took me a lot of time to fix it, the patch was badly garbled. May be broken formatting everywhere in the patch was also caused by this. Then, I didn't see any tests in the patch, not even syntax tests, but also not create field, store, read tests. Please add them. Now, the review: > diff --git a/sql/field.cc b/sql/field.cc > --- a/sql/field.cc > +++ b/sql/field.cc > @@ -7808,6 +7808,35 @@ uint Field_blob::is_equal(Create_field *new_field) > > #ifdef HAVE_SPATIAL > > +uint gis_field_options_image(uchar *buff, List<Create_field> &create_fields) > +{ > + uint image_size= 0; > + uint n_field= 0; > + List_iterator<Create_field> it(create_fields); > + Create_field *field; > + while ((field= it++)) > + { > + if (field->sql_type != MYSQL_TYPE_GEOMETRY) > + continue; > + if (buff) > + int2store(buff + image_size, ((uint16) n_field)); > + image_size+= 2; > + if (buff) > + int2store(buff + image_size, ((uint16) field->length)); > + image_size+= 2; > + if (buff) > + int2store(buff + image_size, ((uint16) field->decimals)); > + image_size+= 2; > + if (buff) > + int4store(buff + image_size, ((uint32) field->srid)); > + image_size+= 4; > + n_field++; > + } > + > + return image_size; > +} Hmm... So, you basically, put an array of properties in frm, under EXTRA2_GIS Two comments: 1. Why n_field, length and decimals? I thought you only add srid, other properties are already stored elsewhere, aren't they? 2. Really think about upgrades downgrades. The "extra" section in the frm doesn't handle them well, I know how these problems happen. Think, what happens when you add more properties here? What if you remove some? How new server will parse old frms, how old server will read new frms? > + > + > void Field_geom::sql_type(String &res) const > { > CHARSET_INFO *cs= &my_charset_latin1; > @@ -9181,7 +9210,7 @@ bool Create_field::init(THD *thd, char *fld_name, enum_field_types fld_type, > Item *fld_on_update_value, LEX_STRING *fld_comment, > char *fld_change, List<String> *fld_interval_list, > CHARSET_INFO *fld_charset, uint fld_geom_type, > - Virtual_column_info *fld_vcol_info, > + char *fld_srid, Virtual_column_info *fld_vcol_info, fix the formatting, please > engine_option_value *create_opt, bool check_exists) > { > uint sign_len, allowed_type_modifier= 0; > @@ -9232,6 +9261,7 @@ bool Create_field::init(THD *thd, char *fld_name, enum_field_types fld_type, > pack_length= key_length= 0; > charset= fld_charset; > geom_type= (Field::geometry_type) fld_geom_type; > + srid= fld_srid ? (uint)atoi(fld_srid) : 0; and here > interval_list.empty(); > > comment= *fld_comment; > @@ -9635,7 +9665,7 @@ Field *make_field(TABLE_SHARE *share, uchar *ptr, uint32 field_length, > uint pack_flag, > enum_field_types field_type, > CHARSET_INFO *field_charset, > - Field::geometry_type geom_type, > + Field::geometry_type geom_type, uint srid, and here, and everywhere, I won't comment on that anymore > Field::utype unireg_check, > TYPELIB *interval, > const char *field_name) > diff --git a/sql/sql_lex.h b/sql/sql_lex.h > --- a/sql/sql_lex.h > +++ b/sql/sql_lex.h > @@ -2358,6 +2358,7 @@ struct LEX: public Query_tables_list > char *backup_dir; /* For RESTORE/BACKUP */ > char* to_log; /* For PURGE MASTER LOGS TO */ > char* x509_subject,*x509_issuer,*ssl_cipher; > + char *srid; really? it can only be used for field declarations, I don't think there's a need to put it in LEX see below. > String *wild; /* Wildcard in SHOW {something} LIKE 'wild'*/ > sql_exchange *exchange; > select_result *result; > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -6359,6 +6360,16 @@ precision: > } > ; > > +srid_option: > + /* empty */ > + { Lex->srid= (char *)0; } > + | > + SRID_SYM EQ NUM > + { > + Lex->srid=$3.str; > + } return the value of srid here, don't store it in lex. Like $$ = $3.str; or even $$ = atoi($3.str); > + ; > + > field_options: > /* empty */ {} > | field_opt_list {} > diff --git a/sql/table.cc b/sql/table.cc > --- a/sql/table.cc > +++ b/sql/table.cc > @@ -1451,6 +1464,20 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, > #ifdef HAVE_SPATIAL > geom_type= (Field::geometry_type) strpos[14]; > charset= &my_charset_bin; > + if (gis_options && gis_options_len >= 10) > + { > + gis_length= uint2korr(gis_options+2); > + gis_decimals= uint2korr(gis_options+4); > + srid= uint4korr(gis_options+6); > + gis_options_len-= 10; > + gis_options+= 10; If you have the function to generate this data in field.cc, I'd put a small function to read it in field.cc too. Just to have all reading/writing code in the same place. > + } > + else > + { > + srid= 0; > + gis_length= 0; > + gis_decimals= 0; > + } > #else > goto err; > #endif > diff --git a/sql/unireg.h b/sql/unireg.h > --- a/sql/unireg.h > +++ b/sql/unireg.h > @@ -190,6 +190,7 @@ enum extra2_frm_value_type { > #define EXTRA2_ENGINE_IMPORTANT 128 > > EXTRA2_ENGINE_TABLEOPTS=128, > + EXTRA2_GIS=129, better make it 2, not 129 (below EXTRA2_ENGINE_IMPORTANT threshold), because you de facto ignore it anyway when HAVE_SPATIAL is not defined. > }; > > int rea_create_table(THD *thd, LEX_CUSTRING *frm, > diff --git a/unittest/mysys/CMakeLists.txt b/unittest/mysys/CMakeLists.txt > --- a/unittest/mysys/CMakeLists.txt > +++ b/unittest/mysys/CMakeLists.txt > @@ -17,7 +17,7 @@ MY_ADD_TESTS(bitmap base64 my_vsnprintf my_atomic my_rdtsc lf my_malloc > LINK_LIBRARIES mysys) > > MY_ADD_TESTS(ma_dyncol > - LINK_LIBRARIES mysqlclient) > + LINK_LIBRARIES mysqlclient stdc++) Why? > > IF(WIN32) > MY_ADD_TESTS(my_delete LINK_LIBRARIES mysys) > Regards, Sergei

2 2

[Maria-developers] Fwd: [debian-mysql] Bug#766996: libmariadb-client-lgpl-dev-compat: misses a mysql_config symlink
by Daniel Schepler 10 Nov '14

10 Nov '14

I recently got the bug report below for the MariaDB C client library (Debian package). The request to add a compatibility symlink from mysql_config to mariadb_config makes sense to me -- do you see any potential problems with adding it? -- Daniel Schepler ---------- Forwarded message ---------- From: Rene Engelhard <rene(a)debian.org> Date: Mon, Oct 27, 2014 at 6:48 AM Subject: [debian-mysql] Bug#766996: libmariadb-client-lgpl-dev-compat: misses a mysql_config symlink To: Debian Bug Tracking System <submit(a)bugs.debian.org> Package: libmariadb-client-lgpl-dev-compat Version: 2.0.0-1 Severity: important https://packages.debian.org/sid/libmariadb-client-lgpl-dev-compat says: "MariaDB Client Library for C, compatibility symlinks MariaDB is a fast, stable and true multi-user, multi-threaded SQL database server. SQL (Structured Query Language) is the most popular database query language in the world. The main goals of MariaDB are speed, robustness and ease of use. This package includes compatibility symlinks to allow sources expecting the MySQL client libraries to be built against the MariaDB Client Library for C." The full contents are: https://packages.debian.org/sid/amd64/libmariadb-client-lgpl-dev-compat/fil… so it won't work with packages expecting mysql_config to give them the flags -> it should also ship a mysql_config -> mariadb_config link. Regards, Rene _______________________________________________ pkg-mysql-maint mailing list pkg-mysql-maint(a)lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-mysql-maint

2 1

Re: [Maria-developers] [Commits] Rev 4343: MDEV-6179: dynamic columns functions/cast()/convert() doesn't play nice with CREATE/ALTER TABLE in lp:~maria-captains/maria/5.5
by Oleksandr Byelkin 10 Nov '14

10 Nov '14

Hi, Sergei! Thank you! The patch is OK. On 08.11.14 19:54, Sergei Golubchik wrote: > At lp:~maria-captains/maria/5.5 > > ------------------------------------------------------------ > revno: 4343 > revision-id: sergii(a)pisem.net-20141108185442-37i07fh0bvm2atjr > parent: sergii(a)pisem.net-20141102163302-8ukz7pi4mwsx89n0 > committer: Sergei Golubchik <sergii(a)pisem.net> > branch nick: 5.5 > timestamp: Sat 2014-11-08 19:54:42 +0100 > message: > MDEV-6179: dynamic columns functions/cast()/convert() doesn't play nice with CREATE/ALTER TABLE > > When parsing a field declaration, grab type information from LEX before it's overwritten > by further rules. Pass type information through the parser stack to the rule that needs it. > === modified file 'mysql-test/r/create.result' > --- a/mysql-test/r/create.result 2013-05-11 17:31:50 +0000 > +++ b/mysql-test/r/create.result 2014-11-08 18:54:42 +0000 > @@ -2430,3 +2430,50 @@ a b > 1 1 > unlock tables; > drop table t1,t2; > +# > +# MDEV-6179: dynamic columns functions/cast()/convert() doesn't > +# play nice with CREATE/ALTER TABLE > +# > +create table t1 ( > +color char(32) as (COLUMN_GET(dynamic_cols, 1 as char)) persistent, > +cl char(32) as (COLUMN_GET(COLUMN_ADD(COLUMN_CREATE(1 , 'blue' as char), 2, 'ttt'), i as char)) persistent, > +item_name varchar(32) primary key, -- A common attribute for all items > +i int, > +dynamic_cols blob -- Dynamic columns will be stored here > +); > +INSERT INTO t1(item_name, dynamic_cols, i) VALUES > +('MariaDB T-shirt', COLUMN_CREATE(1, 'blue', 2, 'XL'), 1); > +INSERT INTO t1(item_name, dynamic_cols, i) VALUES > +('Thinkpad Laptop', COLUMN_CREATE(1, 'black', 3, 500), 2); > +select item_name, color, cl from t1; > +item_name color cl > +MariaDB T-shirt blue blue > +Thinkpad Laptop black ttt > +show create table t1; > +Table Create Table > +t1 CREATE TABLE `t1` ( > + `color` char(32) AS (COLUMN_GET(dynamic_cols, 1 as char)) PERSISTENT, > + `cl` char(32) AS (COLUMN_GET(COLUMN_ADD(COLUMN_CREATE(1 , 'blue' as char), 2, 'ttt'), i as char)) PERSISTENT, > + `item_name` varchar(32) NOT NULL, > + `i` int(11) DEFAULT NULL, > + `dynamic_cols` blob, > + PRIMARY KEY (`item_name`) > +) ENGINE=MyISAM DEFAULT CHARSET=latin1 > +drop table t1; > +create table t1 ( > +n int, > +c char(32) as (convert(cast(n as char), char)) persistent > +); > +insert into t1(n) values (1),(2),(3); > +select * from t1; > +n c > +1 1 > +2 2 > +3 3 > +show create table t1; > +Table Create Table > +t1 CREATE TABLE `t1` ( > + `n` int(11) DEFAULT NULL, > + `c` char(32) AS (convert(cast(n as char), char)) PERSISTENT > +) ENGINE=MyISAM DEFAULT CHARSET=latin1 > +drop table t1; > > === modified file 'mysql-test/t/create.test' > --- a/mysql-test/t/create.test 2012-12-17 20:34:56 +0000 > +++ b/mysql-test/t/create.test 2014-11-08 18:54:42 +0000 > @@ -2021,3 +2021,35 @@ connection default; > select * from t1; > unlock tables; > drop table t1,t2; > + > +--echo # > +--echo # MDEV-6179: dynamic columns functions/cast()/convert() doesn't > +--echo # play nice with CREATE/ALTER TABLE > +--echo # > +create table t1 ( > + color char(32) as (COLUMN_GET(dynamic_cols, 1 as char)) persistent, > + cl char(32) as (COLUMN_GET(COLUMN_ADD(COLUMN_CREATE(1 , 'blue' as char), 2, 'ttt'), i as char)) persistent, > + item_name varchar(32) primary key, -- A common attribute for all items > + i int, > + dynamic_cols blob -- Dynamic columns will be stored here > +); > +INSERT INTO t1(item_name, dynamic_cols, i) VALUES > + ('MariaDB T-shirt', COLUMN_CREATE(1, 'blue', 2, 'XL'), 1); > +INSERT INTO t1(item_name, dynamic_cols, i) VALUES > + ('Thinkpad Laptop', COLUMN_CREATE(1, 'black', 3, 500), 2); > + > +select item_name, color, cl from t1; > +show create table t1; > + > +drop table t1; > + > +create table t1 ( > + n int, > + c char(32) as (convert(cast(n as char), char)) persistent > +); > +insert into t1(n) values (1),(2),(3); > + > +select * from t1; > +show create table t1; > + > +drop table t1; > > === modified file 'sql/sql_lex.h' > --- a/sql/sql_lex.h 2014-08-02 19:26:16 +0000 > +++ b/sql/sql_lex.h 2014-11-08 18:54:42 +0000 > @@ -105,6 +105,15 @@ struct sys_var_with_base > LEX_STRING base_name; > }; > > +struct LEX_TYPE > +{ > + enum enum_field_types type; > + char *length, *dec; > + CHARSET_INFO *charset; > + void set(int t, char *l, char *d, CHARSET_INFO *cs) > + { type= (enum_field_types)t; length= l; dec= d; charset= cs; } > +}; > + > #ifdef MYSQL_SERVER > /* > The following hack is needed because mysql_yacc.cc does not define > > === modified file 'sql/sql_yacc.yy' > --- a/sql/sql_yacc.yy 2014-08-02 19:26:16 +0000 > +++ b/sql/sql_yacc.yy 2014-11-08 18:54:42 +0000 > @@ -737,6 +737,7 @@ static bool add_create_index (LEX *lex, > LEX_STRING lex_str; > LEX_STRING *lex_str_ptr; > LEX_SYMBOL symbol; > + LEX_TYPE lex_type; > Table_ident *table; > char *simple_string; > Item *item; > @@ -1471,13 +1472,15 @@ bool my_yyoverflow(short **a, YYSTYPE ** > %type <string> > text_string opt_gconcat_separator > > +%type <lex_type> field_def > + > %type <num> > type type_with_opt_collate int_type real_type order_dir lock_option > udf_type if_exists opt_local opt_table_options table_options > table_option opt_if_not_exists opt_no_write_to_binlog > opt_temporary all_or_any opt_distinct > opt_ignore_leaves fulltext_options spatial_type union_option > - start_transaction_opts field_def > + start_transaction_opts > union_opt select_derived_init option_type2 > opt_natural_language_mode opt_query_expansion > opt_ev_status opt_ev_on_completion ev_on_completion opt_ev_comment > @@ -5483,11 +5486,11 @@ field_spec: > field_def > { > LEX *lex=Lex; > - if (add_field_to_list(lex->thd, &$1, (enum enum_field_types) $3, > - lex->length,lex->dec,lex->type, > + if (add_field_to_list(lex->thd, &$1, $3.type, > + $3.length, $3.dec, lex->type, > lex->default_value, lex->on_update_value, > &lex->comment, > - lex->change,&lex->interval_list,lex->charset, > + lex->change, &lex->interval_list, $3.charset, > lex->uint_geom_type, > lex->vcol_info, lex->option_list)) > MYSQL_YYABORT; > @@ -5495,13 +5498,15 @@ field_spec: > ; > > field_def: > - type opt_attribute {} > - | type opt_generated_always AS '(' virtual_column_func ')' > - vcol_opt_specifier > - vcol_opt_attribute > - { > - $$= (enum enum_field_types)MYSQL_TYPE_VIRTUAL; > - Lex->vcol_info->set_field_type((enum enum_field_types) $1); > + type opt_attribute > + { $$.set($1, Lex->length, Lex->dec, Lex->charset); } > + | type opt_generated_always AS > + { $<lex_type>$.set($1, Lex->length, Lex->dec, Lex->charset); } > + '(' virtual_column_func ')' vcol_opt_specifier vcol_opt_attribute > + { > + $$= $<lex_type>4; > + Lex->vcol_info->set_field_type($$.type); > + $$.type= (enum enum_field_types)MYSQL_TYPE_VIRTUAL; > } > ; > > @@ -6892,11 +6897,11 @@ alter_list_item: > { > LEX *lex=Lex; > if (add_field_to_list(lex->thd,&$3, > - (enum enum_field_types) $5, > - lex->length,lex->dec,lex->type, > + $5.type, > + $5.length, $5.dec, lex->type, > lex->default_value, lex->on_update_value, > &lex->comment, > - $3.str, &lex->interval_list, lex->charset, > + $3.str, &lex->interval_list, $5.charset, > lex->uint_geom_type, > lex->vcol_info, lex->option_list)) > MYSQL_YYABORT; > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

Re: [Maria-developers] MDEV-6179: dynamic columns functions/cast()/convert() doesn't play nice with CREATE/ALTER TABLE in file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6179/
by Sergei Golubchik 08 Nov '14

08 Nov '14

Hi, Sanja! On Nov 02, sanja(a)askmonty.org wrote: > At file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6179/ > > ------------------------------------------------------------ > revno: 4341 > revision-id: sanja(a)askmonty.org-20141102133114-v46li6r2goyzhhp7 > parent: sanja(a)askmonty.org-20141031130729-dya4hci39x6zrv89 > committer: sanja(a)askmonty.org > branch nick: work-maria-5.5-MDEV-6179 > timestamp: Sun 2014-11-02 14:31:14 +0100 > message: > MDEV-6179: dynamic columns functions/cast()/convert() doesn't play > nice with CREATE/ALTER TABLE > > Type attributes (length, dec, charset) which parsed to global > variables now saved and restored during prsing types which can > appeared in a expression. Why did you create a stack in LEX, when bison generated parser already has a stack of states? Also, there's no need to store that in LEX, it doesn't need to live longer than the field_spec rule. By coincidence I've spent a couple of last days rewriting this very part of the parser. I've also fixed this bug without knowing it :) But in 10.1, and the patch is too big to put it in 10.0, let alone 5.5, so it's useless here. Still, I tried to create a smaller patch for this bug in 5.5. Please, review, it's on the commit list. Regards, Sergei

1 0

Re: [Maria-developers] MariaDB 10.0.14 got rid of replication safety warning without warning in release notes?
by Pavel Ivanov 08 Nov '14

08 Nov '14

And here's response from Monty to my last email. And this response together with the whole thread makes me really worried. It alludes to the fact that all MySQL and MariaDB developers before making any changes should read the whole codebase first and then make sure that their changes work the same way and that they conform to the intentions of original code developers even though there's no way they can talk to those original developers. AFAIK that's not how development on big projects with hundreds of participants works. And it really raises the question for me: what kind of quality the product that is developed with such principles in mind can have? Sergei Golubchik, I believe you are the technical leader of MariaDB project. Could you please tell me what you think about this issue? On Tue, Nov 4, 2014 at 6:32 AM, Michael Widenius <michael.widenius(a)gmail.com> wrote: > > Hi! > >>>>>> "Pavel" == Pavel Ivanov <pivanof(a)google.com> writes: > > Pavel> On Sun, Nov 2, 2014 at 3:27 PM, Michael Widenius <monty(a)mariadb.org> wrote: >>>>> This is the same order that keys are presented in SHOW CREATE TABLE >>> > Pavel> Why is it safe to assume that keys have the same order on master and on slave? >>> >>> Primary and Unique keys should always be ordered the same way on >>> master and slave. This is handled by the 'sort_keys()' function that >>> you can find at: sql/sql_table.cc, line 2770. > > Pavel> This function seem to sort only by "type" of the key. If several > Pavel> unique keys have the same "type" according to its classification then > Pavel> they will be in the "original order" as the comment in the function > Pavel> says. And the original order may be different. > > This function ensures that the table will be identical on slave and > master as long as one of the following requirements are fulfilled: > > - You are using the same create table and alter table statements on > both machines. > - You are using the same CREATE TABLE statement as 'show create' shows > on the master. > - You are adding unique keys in the same order on master and slave. > > This is always the case when you are using the replication that MySQL > and MariaDB uses. > > If you are creating tables differently on the slave than on the > master, then you are on your own. > > So in all normal and most abnormal cases, the table will be created > identically on the master and slave, even if the slave is using a > different storage engine, have different columns or have additional > keys than on the master. > > <cut> > >>> I added a test for this to: >>> mysql-test/t/insert.test >>> >>> --echo # >>> --echo # MDEV-5168: Ensure that we can disable duplicate key warnings >>> --echo # from INSERT IGNORE >>> --echo # > > Pavel> This test doesn't check which duplicate key is asserted to execute ON > Pavel> DUPLICATE KEY action. > > This test verifies that we don't get warnings for the tests. > Elinas test verifies which key is used. > >>> Elena added another test for this in >>> mysql-test/suite/storage_engine/insert_with_keys.test >>> >>> See line 142, that start with: >>> --let $create_definition = a $int_indexed_col UNIQUE KEY, b $int_indexed_col UNIQUE KEY, c $int_col > > Pavel> This test seem to check that UPDATE is executed on the correct > Pavel> duplicate key, but it's not executed during regular mtr run. How and > Pavel> when it's executed in MariaDB? > > The storage engine tests are to be used by people that implements > their own storage engine for MariaDB. > > We run the extended tests from time to time to verify storage engines. > We will also run it on some buildbot servers when running releae builds. > The storage engine tests is not something that has to be run for > normal builds. > >>> What is important is that all storage engine should work the same >>> way. If one returns something different compared to any of the >>> existing engines then it's a bug. >>> >>> In this case the implicit documentation is how MyISAM, Aria and InnoDB >>> works. All of them returns the first conflicting key. > > Pavel> I'm sorry to say, but you've explained exactly the reason why this > Pavel> behavior is not guaranteed and can be changed at any time. > > It can only be changed if someone adds new code that breaks the > storage engine interface, as defined by existing engines and one > doesn't check it with the storage engine test suite. > > Pavel> Yes, I > Pavel> understand that when you wrote the interface you had some particular > Pavel> requirements in mind. But ultimately it's up to people actively > Pavel> working on the code (who I believe are counted by hundreds now) to > Pavel> decide whether to maintain the particular behavior or change > it. > > It's the MariaDB captains that are responsible to maintain the > interface. The only way one can change something is to propose a > change and get it accepted. > > It's not free for anyone to just change old functionality and expect > that to be accepted in the standard version. > > In all cases, including this, it's an absolute requirement that > engines MUST work exactly as existing engines, so that a user can > switch engines and expect them to work identically. > > (The only exceptions are things that are impossible to fix, like > the differences between transactional and not transactional tables and > that there is no defined row order in tables). > > If engine writes don't ensure this, then you will get data corruption > on the server by just running with different engines. > > For case on insert on duplicate key, there is no acceptable reason > for engines to work differently. > > Pavel> And > Pavel> as nobody knows the original requirements. > > The original requirements was well defined. Just that non one bother > to ask doesn't mean that the requirement didn't exist. > > And the fact that all existing engines in MariaDB ensures this shows > that the requirements has been understood. > > In the case when we notice that two engines works differently, it's up > to the MariaDB captains to fix this! > The fix is NOT to extend the API and allow things to work differently. > > > Pavel> and the reasons why they > Pavel> can't be changed, and when the behavior is not tested anywhere > > We add new tests when we notice that they are needed, as we did in > this case. > For example, the case with INSERT ... ON DUPLICATE KEY is now tested. > > Pavel> and > Pavel> doesn't have any documentation even in the weakest form of comments or > Pavel> asserts, the decision is usually very easy -- change at will, nobody > Pavel> should ever depend on implementation details. > > Yes, no one should depend in implementation details, except when the > implementation detail is different from other engines and it results > in different data. > > If that is the case then the engine that does things differently than > the existing engines must be changed. > > Pavel> Even if you promise to > Pavel> fix storage engines that change this behavior, by the time you notice > Pavel> the change you'll likely won't be able to fix it (as e.g. in case of > Pavel> https://mariadb.atlassian.net/browse/MDEV-6916) or someone will > Pavel> already experience a data corruption. > > In most cases we can fix this. What was unfortunate with the views > code was that we didn't store the MariaDB version number in the view > definition. This is to be fixed in 10.1 so that this will never happen > again! > > Pavel> I'd strongly recommend to revert this change. > > Sorry, but I can't agree to revert the change because: > > a) Something that can't ever happen in MariaDB (when used correctly > with the including engines). > b) Allowing a behavior that could for the same command result in > different data sets, depending on the storage engine. > c) I have not seen a real world > > One of my main jobs is to ensure that the storage engine API is > evolved in a resonable manner and if there is a conflict between how > storage engine works ensure that things are resolved so that the user > can expect the same results independent on the storage engine. > > Reverting the change would go directly agains the above and also > against what I belive is the future of the storage engine API. > > Regards, > Monty

2 1

Re: [Maria-developers] MariaDB 10.0.14 got rid of replication safety warning without warning in release notes?
by Pavel Ivanov 08 Nov '14

08 Nov '14

Monty, can you please subscribe all your email address to maria-develoeprs@ ? People reading the list cannot see your emails. I'm forwarding here the unaltered message from Monty that I was answering to in my last email. On Sun, Nov 2, 2014 at 3:27 PM, Michael Widenius <monty(a)mariadb.org> wrote: > > Hi! > > I have again to apologize for a late reply. > After my last reply to your topic I traveled to China and noticed > that the firewall there has got much better; You can't access google, > twitter, facebook, blogspot etc :( I tried to connect over a VPN, but > that didn't work for me either. > > It did feel a bit strange to not be able to do computer work for more > than a week, but the lovely people in China who took care of me did > compensate for that. > >>>>>> "Pavel" == Pavel Ivanov <pivanof(a)google.com> writes: > > Pavel> Which key is "the first duplicate key". >>> >>> The first key that has a duplicate value in the order of keys in the KEY >>> structure that MariaDB provides for the storage engine. > > Pavel> I'm definitely not that familiar with the relevant MariaDB code, but I > Pavel> didn't find KEY structure to be passed to any insert-related function. > Pavel> Can you elaborate (preferably with code references) how this > Pavel> restriction is supposed to work? > > The following code examples are from 10.0. > > All handler functions get access to the TABLE object, which defines > the record format and the keys. > > The keys are stored in TABLE->key_info (see table.h, line 1062) > > When one reads a key, one refers to keys according to the number in > the key_info structure. For example: > > A set of reads by key is done by calling: > > handler->ha_index_init(key_number,..); > read_until_no_more_rows > handler->ha_index_read_map(). > handler->ha->index_end() > > When one calls handler->info() to ask for a which key caused a > duplicate error, it's supposed to store the first duplicate key (key > with smallest number) in handler->errkey. > >>> This is the same order that keys are presented in SHOW CREATE TABLE > > Pavel> Why is it safe to assume that keys have the same order on master and on slave? > > Primary and Unique keys should always be ordered the same way on > master and slave. This is handled by the 'sort_keys()' function that > you can find at: sql/sql_table.cc, line 2770. > > The only case when this is not the case, is if you add an unique key > on the slave that was not on the master and there is conflicts when > you insert things because of this key. > > However, if you do this, I would say that you are on your own. > >>> To help engine writes we are constantly adding new test in the storage >>> engine test suite, to make it easier for storage engine vendors to >>> test that their storage engine follows the interface. >>> We have now added a test to check for the correct behaviour for the >>> case of INSERT ... ON DUPLICATE KEY. > > Pavel> This is very interesting. Can you say which test it is and what > Pavel> exactly does it test? Is it present in 10.0.14 or it's only at the > Pavel> latest 10.0 branch? > > I added a test for this to: > mysql-test/t/insert.test > > --echo # > --echo # MDEV-5168: Ensure that we can disable duplicate key warnings > --echo # from INSERT IGNORE > --echo # > > Elena added another test for this in > mysql-test/suite/storage_engine/insert_with_keys.test > > See line 142, that start with: > --let $create_definition = a $int_indexed_col UNIQUE KEY, b $int_indexed_col UNIQUE KEY, c $int_col > >>> Do you have or use a storage engine that doesn't provide the expected value >>> for the conflicting key ? > > Pavel> I don't have a specific example of storage engine that detects > Pavel> duplicate key in different order, but I'm not convinced and don't have > Pavel> hard proof that such engine cannot exist and that InnoDB itself is not > Pavel> susceptible to this problem. I can easily imagine though that even > Pavel> given the particular order of keys in table definition the key > Pavel> reported as having duplicate can depend on the way storage engine > Pavel> stores those keys. > > As key order is defined on the upper level, above InnoDB, this should > be 100 % safe (at least for InnoDB). > > For other store engines, this is safe for any engine that test for > duplicate key in key order. > > If we ever find a case where it's not safe with any storage engine > that we deliver with MariaDB, we will fix the storage engine. > > It's ok if they do thing differently so that they can detect duplicate > key for other keys faster. > > However, it has been required from all storage engines from start that > if one calls 'handler->info()' to request which key was duplicated, > they must return the first key. > > If not, they are breaking the storage engine interface (both for MySQL > and MariaDB) as there is no way the upper level can guarantee that > things will work identically between storage engines. > > Pavel> BTW, can you point to MySQL documentation and/or comment in the code > Pavel> that indicates this requirement to storage engine implementation? > > Most of the storage engine interface is not 100 % documented. This is > one of the case. > However, I wrote the code related to errkey and this is how it was > supposed to work. > In general if something is not documented, then it's up to the people > that created the interface or are actively working on it to define how > it should work. > > What is important is that all storage engine should work the same > way. If one returns something different compared to any of the > existing engines then it's a bug. > > In this case the implicit documentation is how MyISAM, Aria and InnoDB > works. All of them returns the first conflicting key. > > Pavel> On the related note Release Notes mention the error log flood > Pavel> protection. Why it's not behind flag? How should I disable the feature > Pavel> and get back to the old behavior? > >>> We already had in MariaDB flood protection for some error >>> messages. > > Pavel> Can you say exactly which error messages are suppressed already? Is > Pavel> there a documentation link about that? > > All 'unsafe logging' errors are suppressed. > See enum enum_binlog_stmt_unsafe in sql_lex.h > (This is what I know). > >>> I am happy to look into adding a flag that is usable for any kind of >>> suppressed messages. >>> >>> A couple of ways to do this: >>> >>> a) Not suppress anything if you give --log-warnings a value > 2 >>> b) make LIMIT_UNSAFE_WARNING_ACTIVATION_TIMEOUT a configurable >>> variable, where 0 would mean no suppression. >>> >>> Which of the above, or both, would you prefer? >>> Or do you have any other suggestion of how to do this? > > Pavel> Reading documentation on --log-warnings I don't see how it can be > Pavel> related to the error suppression. So I think making > Pavel> LIMIT_UNSAFE_WARNING_ACTIVATION_TIMEOUT configurable would be much > Pavel> better, though it should apply to all possible error suppressions. > > Currently we only have suppression for a limited set of 'unsafe' > errors, but I agree that when we add a flag we should name this with > a general name to be able to reuse it for all possible future > suppressions. > > In 10.1 we are looking at adding a feature that one can suppress any > error from the error log, but that's a different topic. > > Regards, > Monty

1 0

[Maria-developers] Maybe a new information schema table for replication?
by Roberto Spadim 06 Nov '14

06 Nov '14

Hi guys, i got one question these days and i don't know how to solve... the question is: "How to see the queries executing by replication thread?" i'm thinking about a information schema plugin to see whats is 'queued' in replication thread, at "master" (write to binlog) and "slave" (executing from binlog) side any idea if this is possible? i didn't checked or read any part of binlog/replication code yet thanks guys! :) -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

[Maria-developers] pcre.h should be installed if MariaDB is built with bundled pcre ?
by Kazuhiko Shiozaki 06 Nov '14

06 Nov '14

Hi, pcre.h is not installed by 'make install' even if MariaDB is built with bundled pcre. But pcre.h is referred in one of installed header files like : include/mysql/private/item_cmpfunc.h:#include "pcre.h" /* pcre header file */ So I think that it is better to install pcre.h as well if MariaDB is built with bundled pcre. What do you think ? -- Kazuhiko Shiozaki, Nexedi SA Senior Consultant Nexedi: Consulting and Development of Free / Open Source Software http://www.nexedi.com/ ERP5: Full Featured High End Open Source ERP http://www.erp5.com/

2 1

[Maria-developers] Fwd: Is it soon 'end of this year'?
by Peter Laursen 04 Nov '14

04 Nov '14

Forwarding to MariaDB developers mailing list. It may be a matter for somebody else than Sergei (Georg, for instance) (and "here" means"Webyog"). Abishek detailed errors in a chat with me "symbols like Snippet mysql_client_find_plugin , Snippet mysql_client_register_plugin are not found" ..exactly like in the MySQL bug report. Do you need a JIRA report with a full test case? -- Peter ---------- Forwarded message ---------- From: Abhishek Kumar Pandey <abhishek.pandey(a)webyog.com> Date: Tue, Nov 4, 2014 at 12:51 PM Subject: Re: [Maria-developers] Is it soon 'end of this year'? To: Peter Laursen <peter_laursen(a)webyog.com> Cc: "team. sqlyog" <team.sqlyog(a)webyog.com>, serg(a)mariadb.org, "team.monyog" <team.monyog(a)webyog.com> I am the developer here working with this. I am facing same problem as reported here- http://bugs.mysql.com/bug.php?id=69204 <http://www.google.com/url?q=http%3A%2F%2Fbugs.mysql.com%2Fbug.php%3Fid%3D69…> . Is the same fix needed in the mariadb library? On Tue, Oct 14, 2014 at 6:58 PM, Peter Laursen <peter_laursen(a)webyog.com> wrote: > You will get to this page: https://edelivery.oracle.com/ > > I think you should use a non-webyog email address. Not sure what Mahesh > did. He downloaded frequently in order to check latest MEM. > > > --Peter > > On Tue, Oct 14, 2014 at 3:26 PM, Peter Laursen <peter_laursen(a)webyog.com> > wrote: > >> You could also try to add the code as described by Sergei. First step is >> if it will compile and connect to servers not usinglain text/LDAP. >> >> -- Peter >> >> On Tue, Oct 14, 2014 at 3:12 PM, Supratik Goswami < >> supratik.goswami(a)webyog.com> wrote: >> >>> On Tue, Oct 14, 2014 at 4:58 PM, Sudhindra Bhat < >>> sudhindra.bhat(a)webyog.com> wrote: >>> >>>> +supratik >>>> >>>> We will check this once we have access to MySQL Enterprise server from >>>> Oracle. >>>> >>>> @supratik, >>>> Do we have the binary for MySQL Enterprise server? A Trial is also >>>> sufficient. >>>> >>> >>> No, I think. You may download it. >>> >>>> >>>> On Tue, Oct 14, 2014 at 2:52 PM, Peter Laursen < >>>> peter_laursen(a)webyog.com> wrote: >>>> >>>>> We need to try this again. I don't rememebr who gave up supporting >>>>> 'clear text plugin' with MariaDB API back in April 2013. Probably it was >>>>> not Sudhindra, Likely Shubhansh or Vishal. @Vishal: do you remember >>>>> anything? >>>>> >>>>> I remember one detail from the old discussion we had with MariaDB. >>>>> They told something like "Once enabled nothing special need to be done to >>>>> support clear text. The client wll automatically detect when the server >>>>> requests the password sent as clear text". This mean as far as I can >>>>> understand that we will - unlke early 11.x version - not need a checkbox in >>>>> the connection manager. >>>>> >>>>> We also need access to a commercial server (TRIAL) from Oracle. Mahesh >>>>> could arrange that. Who can today? >>>>> >>>>> >>>>> -- Peter >>>>> >>>>> >>>>> ---------- Forwarded message ---------- >>>>> From: Sergei Golubchik <serg(a)mariadb.org> >>>>> Date: Mon, Oct 13, 2014 at 11:24 PM >>>>> Subject: Re: [Maria-developers] Is it soon 'end of this year'? >>>>> To: Peter Laursen <peter_laursen(a)webyog.com> >>>>> >>>>> >>>>> Hi, Peter! >>>>> >>>>> On Oct 13, Peter Laursen wrote: >>>>> > Thanks .. you probabaly know that I am not a developer rmyself (I >>>>> basically >>>>> > stopped being so when ALGOL died 35 years ago!). I have shared all >>>>> > information with SQLyog developer team and we will look into this >>>>> again. >>>>> >>>>> Sure! Don't hesitate to tell if something won't work for you. >>>>> >>>>> Regards, >>>>> Sergei >>>>> >>>>> >>>> >>>> >>>> -- >>>> Regards, >>>> Sudhindra Bhat >>>> >>> >>> >> >

2 1

Re: [Maria-developers] MariaDB 10.0.14 got rid of replication safety warning without warning in release notes?
by Pavel Ivanov 03 Nov '14

03 Nov '14

On Sun, Nov 2, 2014 at 3:27 PM, Michael Widenius <monty(a)mariadb.org> wrote: >>> This is the same order that keys are presented in SHOW CREATE TABLE > > Pavel> Why is it safe to assume that keys have the same order on master and on slave? > > Primary and Unique keys should always be ordered the same way on > master and slave. This is handled by the 'sort_keys()' function that > you can find at: sql/sql_table.cc, line 2770. This function seem to sort only by "type" of the key. If several unique keys have the same "type" according to its classification then they will be in the "original order" as the comment in the function says. And the original order may be different. > The only case when this is not the case, is if you add an unique key > on the slave that was not on the master and there is conflicts when > you insert things because of this key. > > However, if you do this, I would say that you are on your own. > >>> To help engine writes we are constantly adding new test in the storage >>> engine test suite, to make it easier for storage engine vendors to >>> test that their storage engine follows the interface. >>> We have now added a test to check for the correct behaviour for the >>> case of INSERT ... ON DUPLICATE KEY. > > Pavel> This is very interesting. Can you say which test it is and what > Pavel> exactly does it test? Is it present in 10.0.14 or it's only at the > Pavel> latest 10.0 branch? > > I added a test for this to: > mysql-test/t/insert.test > > --echo # > --echo # MDEV-5168: Ensure that we can disable duplicate key warnings > --echo # from INSERT IGNORE > --echo # This test doesn't check which duplicate key is asserted to execute ON DUPLICATE KEY action. > Elena added another test for this in > mysql-test/suite/storage_engine/insert_with_keys.test > > See line 142, that start with: > --let $create_definition = a $int_indexed_col UNIQUE KEY, b $int_indexed_col UNIQUE KEY, c $int_col This test seem to check that UPDATE is executed on the correct duplicate key, but it's not executed during regular mtr run. How and when it's executed in MariaDB? >>> Do you have or use a storage engine that doesn't provide the expected value >>> for the conflicting key ? > > Pavel> I don't have a specific example of storage engine that detects > Pavel> duplicate key in different order, but I'm not convinced and don't have > Pavel> hard proof that such engine cannot exist and that InnoDB itself is not > Pavel> susceptible to this problem. I can easily imagine though that even > Pavel> given the particular order of keys in table definition the key > Pavel> reported as having duplicate can depend on the way storage engine > Pavel> stores those keys. > > As key order is defined on the upper level, above InnoDB, this should > be 100 % safe (at least for InnoDB). > > For other store engines, this is safe for any engine that test for > duplicate key in key order. > > If we ever find a case where it's not safe with any storage engine > that we deliver with MariaDB, we will fix the storage engine. > > It's ok if they do thing differently so that they can detect duplicate > key for other keys faster. > > However, it has been required from all storage engines from start that > if one calls 'handler->info()' to request which key was duplicated, > they must return the first key. > > If not, they are breaking the storage engine interface (both for MySQL > and MariaDB) as there is no way the upper level can guarantee that > things will work identically between storage engines. > > Pavel> BTW, can you point to MySQL documentation and/or comment in the code > Pavel> that indicates this requirement to storage engine implementation? > > Most of the storage engine interface is not 100 % documented. This is > one of the case. > However, I wrote the code related to errkey and this is how it was > supposed to work. > In general if something is not documented, then it's up to the people > that created the interface or are actively working on it to define how > it should work. > > What is important is that all storage engine should work the same > way. If one returns something different compared to any of the > existing engines then it's a bug. > > In this case the implicit documentation is how MyISAM, Aria and InnoDB > works. All of them returns the first conflicting key. I'm sorry to say, but you've explained exactly the reason why this behavior is not guaranteed and can be changed at any time. Yes, I understand that when you wrote the interface you had some particular requirements in mind. But ultimately it's up to people actively working on the code (who I believe are counted by hundreds now) to decide whether to maintain the particular behavior or change it. And as nobody knows the original requirements and the reasons why they can't be changed, and when the behavior is not tested anywhere and doesn't have any documentation even in the weakest form of comments or asserts, the decision is usually very easy -- change at will, nobody should ever depend on implementation details. Even if you promise to fix storage engines that change this behavior, by the time you notice the change you'll likely won't be able to fix it (as e.g. in case of https://mariadb.atlassian.net/browse/MDEV-6916) or someone will already experience a data corruption. I'd strongly recommend to revert this change. Thank you, Pavel

1 0

[Maria-developers] MDEV-5528 Command line variable to choose MariaDB-5.3 vs MySQL-5.6 temporal data formats
by Alexander Barkov 03 Nov '14

03 Nov '14

Hello Ian, I've pushed MDEV-5528 into 10.1. I guess it will need some documenting. Can you please have a look? Thanks.

1 0

[Maria-developers] nginx sources in mroonga
by Alexander Barkov 03 Nov '14

03 Nov '14

Hi Kentoku, Sergei Golubchik is asking why we have complete nginx sources inside mroonga? Can you remove them please? Thanks.

3 2

[Maria-developers] Cross-compiling MariaDB galera cluster
by Sylvain Raybaud 30 Oct '14

30 Oct '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dear MariaDB community, Recently I've been trying to cross-compile MariaDB and I've been running in some problems. I'd like to have your opinion on them. 1) Determining stack direction 2) Building Percona XtraDB 3) Building jemalloc Let me first tell that I'm trying to build MariaDB galera version 10.0.12. I'm compiling for Arm but may be compiling for something else in the unpredictable future. 1) Determining stack direction I started by passing the following options to cmake (via the buildroot system): - -DCMAKE_CROSSCOMPILING=1 -DWITH_WSREP=1 -DWITH_INNODB_DISALLOW_WRITES=1 The first problem I ran into was cmake complaining: STACK_DIRECTION is not defined. Please specify -DSTACK_DIRECTION=1 or -DSTACK_DIRECTION=-1 when calling cmake. It seems that STACK_DIRECTION is not automatically determined when cross-compiling. I may determine the correct value for a given target. But if I need a portable solution (and I need one in order to create a buildroot package so that I can build for many different target without too much hassle) I'm not sure what my options are. It seems that the toolchain is aware of this value since stack direction (or more precisely STACK_GROWS_DOWNWARD) is set in ./output/host/opt/ext-toolchain/lib/gcc/arm-none-linux-gnueabi/4.8.1/plugin/include/config/arm/arm.h Maybe there is a way to use this somehow? What do you think? 2) Building Percona XtraDB For the sake of trying I explicitly set -DSTACK_DIRECTION=1 just to see what happens. Building goes on until it reaches the xtradb module. Then it fails with: CMake Error at storage/xtradb/CMakeLists.txt:434 (MESSAGE): Percona XtraDB is not supported on this platform (Here I'll just copy-paste something I wrote on maria-discuss list a few days ago) This error is triggered in storage/xtradb/CMakeLists.txt when XTRADB_OK is not set; it seems to me that XTRADB_OK is set in either one of the following cases: a) CMAKE_CROSSCOMPILING is NOT set and some tests are successful (thus setting HAVE_IB_GCC_ATOMIC_BUILTINS) or b) We're compiling for MSVC or CMAKE_SYSTEM_NAME is "SunOS" and some other conditions are met. Since I'm not compiling for MSVC or SunOS and I'm crosscompiling indeed, XTRADB_OK is not set and the error is triggered. Here I'm stuck. I've got the feeling that as such one cannot cross-compile xtradb because XTRADB_OK will never be set; or maybe with proper options passed to cmake the tests would be made elsewhere and HAVE_IB_GCC_ATOMIC_BUILTINS would be set beforehand? Or maybe I should patch this CMakeLists.txt? Or maybe I should just compile without xtradb altogether, if that's possible? Finally I just passed -DXTRADB_OK just to see what would happen, again (obviously there is little chance that it would result in anything usable). And reached problem #3: 3) Building jemalloc This one fails at configure time with: checking whether we are cross compiling... configure: error: in `/home/sylvain/DEPOTS/buildroot/output/build/mariadb-galera-10.0.12/extra/jemalloc/build': configure: error: cannot run C compiled programs. If you meant to cross compile, use `--host'. Somehow I was expecting that having passed -DCMAKE_CROSSCOMPILING=1 and the host being set by buildroot it would propagate down to the modules... Am I missing something? At this point I'm out of dirty hacks, even to "see what happens" ;) Cheers, - -- Sylvain Raybaud www.green-communications.fr -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJT4ke/AAoJEEkkwl4JtJ9yJW0QALBz3wj0VsfMtUBUabP13q74 ++y1AiCLZ0js+UGCqxB9PVz4yvhhnMAz8QnPpH37XJtDm+VaJBQTEZL65CHF3nlr dHe46/q/wswlbdhNsQu6JmMe3JoOe2TFbjDenQMiUvW9mOilt86ioveHVLGYbpnq 5LYMrPlYXUBpDcxEYXXbTpIHsAoHUrOE1Gj5AsxwbUvQuHDDr9M/Ws01iIXLbzT8 0pImhgBPYUwqaj9tJnPkls+9LzoOb2m269n9TCbmWk0ixakpHjeNMn5lnRDhOwNh AKkHrGdMUomSBca4G0kQoeNxwC2rCjO8+bRxjh7v6GTdME/sgsgn3yOv2EznldUr tFaO/JfraHSUgmiRyVo41U3uTESxZKRJYOV9IqUhl4HiS85D5oa/bD6l84rhx95J 52/hUoApoStr8tSk0tzXk1/UWJEQ3fUZ7XoXqgWq+c1uMgbgZftAAqAw7Xvmc88t tf69FqBzw5dpf8Kg0I7iAggBpBLIWGJwnlVvcmLY4UUE1cDFfZstMBSXaS155LQF VClK+ytE2Ef8oAB9/nlMXgg4b3WXlD0PRQDRTNIwCAzdvsuu+S0q64xLtw/JYnLl KPcLWfwXy+kRhT+vtUWtqMjjkSQ3e6rJDybK11vgYIAWm/Xfk5s8i0g7Bgmnf3wp JGYUAR285gY0XMqL6LmU =vBo4 -----END PGP SIGNATURE-----

2 4

[Maria-developers] Valgrind failure in Buildbot
by Kristian Nielsen 30 Oct '14

30 Oct '14

Hi Sergey, We briefly discussed the new valgrind error in buildbot after your push on IRC: https://buildbot.askmonty.org/buildbot/builders/work-amd64-valgrind/builds/… ==13161== Conditional jump or move depends on uninitialised value(s) ==13161== at 0x4C280AA: memcpy (mc_replace_strmem.c:115) ==13161== by 0x723F6B: Field_varstring::get_key_image(unsigned char*, unsigned int, Field::imagetype) (field.cc:6932) ==13161== by 0x818D5B: key_copy(unsigned char*, unsigned char*, st_key*, unsigned int, bool) (key.cc:146) ==13161== by 0x6DD6FE: Mrr_ordered_index_reader::interrupt_read() (multi_range_read.cc:448) ==13161== by 0x6DE605: Mrr_ordered_rndpos_reader::refill_from_index_reader() (multi_range_read.cc:688) ==13161== by 0x6DE7D2: Mrr_ordered_rndpos_reader::refill_buffer(bool) (multi_range_read.cc:618) ==13161== by 0x6DFB27: DsMrr_impl::dsmrr_init(handler*, st_range_seq_if*, void*, unsigned int, unsigned int, st_handler_buffer*) (multi_range_read.cc:966) ==13161== by 0x91AAE9: ha_innobase::multi_range_read_init(st_range_seq_if*, void*, unsigned int, unsigned int, st_handler_buffer*) (ha_innodb.cc:13961) ==13161== by 0x6DA60A: JOIN_TAB_SCAN_MRR::open() (sql_join_cache.cc:3867) ==13161== by 0x6D7363: JOIN_CACHE::join_matching_records(bool) (sql_join_cache.cc:2249) ==13161== by 0x6D5647: JOIN_CACHE::join_records(bool) (sql_join_cache.cc:2086) ==13161== by 0x5F8259: sub_select_cache(JOIN*, st_join_table*, bool) (sql_select.cc:16717) ==13161== by 0x6D71BA: JOIN_CACHE::generate_full_extensions(unsigned char*) (sql_join_cache.cc:2397) ==13161== by 0x6D74E6: JOIN_CACHE::join_matching_records(bool) (sql_join_cache.cc:2289) ==13161== by 0x6D5647: JOIN_CACHE::join_records(bool) (sql_join_cache.cc:2086) ==13161== by 0x5F8219: sub_select_cache(JOIN*, st_join_table*, bool) (sql_select.cc:16697) I took a quick look. The problem seems to be that the code is copying out the fields of a key from table->record[0] (key_copy()), however, the first length byte of the value is uninitialised. I think it's the field col_varchar_key in the table, testcase is appended below. I checked a bit back up the callstack. The field col_varchar_key is reported as uninitialised already at the start of Mrr_ordered_rndpos_reader::refill_from_index_reader(), but I'm not really familiar with when and how table->record[0] is supposed to be initialised during multi-range read. As you said on IRC, your recent patch is related to interrupt_read(), so probably best if you take a look and see if you can figure out something. Incidentally, I noticed something that looks wrong in key_copy(to_key, from_record, key_info, key_length, with_zerofill) The comments say that the key is copied from `from_record', however actually the function only works if the from_record passed in is table->record[0] (or more precisely, the same record as is installed in key_info->key_part->field). Because it checks the null bit in from_record, but copies the data from table->record[0]. However, this is not the problem in this particular failure (as in this case from_record == table->record[0]), just struck me as something that could easily cause surprises later... Below is a reduced test case that triggers the valgrind error. It did not trigger for me with a debug build, but _did_ trigger (in 5.5) with this build command: EXTRA_FLAGS="-O3 -fno-omit-frame-pointer -Wno-uninitialized" AM_EXTRA_MAKEFLAGS="VERBOSE=1" BUILD/compile-amd64-valgrind-max - Kristian. ----------------------------------------------------------------------- -- source include/have_xtradb.inc set optimizer_switch='mrr=on,mrr_sort_keys=on,index_condition_pushdown=on'; CREATE TABLE t1 ( pk int(11) NOT NULL AUTO_INCREMENT, col_int_key int(11) NOT NULL, col_varchar_key varchar(1) NOT NULL, col_varchar_nokey varchar(1) NOT NULL, PRIMARY KEY (pk), KEY col_int_key (col_int_key), KEY col_varchar_key (col_varchar_key,col_int_key) ) ENGINE=InnoDB; INSERT INTO t1 VALUES (10,8,'v','v'), (11,8,'f','f'), (12,5,'v','v'), (13,8,'s','s'), (14,8,'a','a'), (15,6,'p','p'), (16,7,'z','z'), (17,2,'a','a'), (18,5,'h','h'), (19,7,'h','h'), (20,2,'v','v'), (21,9,'v','v'), (22,142,'b','b'), (23,3,'y','y'), (24,0,'v','v'), (25,3,'m','m'), (26,5,'z','z'), (27,9,'n','n'), (28,1,'d','d'), (29,107,'a','a'); CREATE TABLE t2 ( pk int(11) NOT NULL AUTO_INCREMENT, col_int_key int(11) NOT NULL, col_varchar_key varchar(1) NOT NULL, col_varchar_nokey varchar(1) NOT NULL, PRIMARY KEY (pk), KEY col_int_key (col_int_key), KEY col_varchar_key (col_varchar_key,col_int_key) ) ENGINE=InnoDB; INSERT INTO t2 VALUES (10,3,'i','i'), (11,186,'x','x'), (12,1,'g','g'), (13,8,'q','q'), (14,226,'m','m'), (15,133,'p','p'), (16,6,'e','e'), (17,3,'t','t'), (18,8,'j','j'), (19,5,'h','h'), (20,7,'w','w'); set join_cache_level=6; set join_buffer_size=1536; SELECT count(*), sum(table1.col_int_key*table2.pk) FROM t2 AS table1, t1 AS table2, t2 AS table3 WHERE table3.col_varchar_nokey = table2.col_varchar_key AND table3.pk > table2.col_varchar_nokey ; -----------------------------------------------------------------------

1 0

Re: [Maria-developers] Rev 4323: Set proper alternative names on RHEL7: they should match Fedora names
by Sergei Golubchik 30 Oct '14

30 Oct '14

Hi, Sergey! I cannot verify whether rhel7 alternative names should match Fedora names or they should be something different. Assuming they should match, your patch is ok. On Oct 09, Sergey Vojtovich wrote: > ------------------------------------------------------------ > revno: 4323 > revision-id: svoj(a)mariadb.org-20141009112748-eq3xwoyobgnz0cxj > parent: svoj(a)mariadb.org-20141008141031-xv2x7vrq269202vf > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 5.5 > timestamp: Thu 2014-10-09 15:27:48 +0400 > message: > Set proper alternative names on RHEL7: they should match Fedora names. > === modified file 'cmake/cpack_rpm.cmake' > --- a/cmake/cpack_rpm.cmake 2014-10-08 07:35:00 +0000 > +++ b/cmake/cpack_rpm.cmake 2014-10-09 11:27:48 +0000 > @@ -182,7 +182,7 @@ IF(RPM MATCHES "(rhel|centos)5") > ELSEIF(RPM MATCHES "(rhel|centos)6") > ALTERNATIVE_NAME("client" "mysql") > ALTERNATIVE_NAME("shared" "mysql-libs") > -ELSEIF(RPM MATCHES "fedora") > +ELSEIF(RPM MATCHES "fedora" OR RPM MATCHES "(rhel|centos)7") > SET(epoch 1) # this is fedora > ALTERNATIVE_NAME("client" "mariadb") > ALTERNATIVE_NAME("client" "mysql") Regards, Sergei

2 2

[Maria-developers] Fwd: [debian-mysql] Backporting the mysql_no_login plugin
by Colin Charles 30 Oct '14

30 Oct '14

This seems to get a +1 for backporting by at least Honza (RH), so I am wondering if we do this in the 5.5 branch too, that is shipping in many distributions. Begin forwarded message: > From: "Norvald H. Ryeng" <norvald.ryeng(a)oracle.com> > Subject: [debian-mysql] Backporting the mysql_no_login plugin > Date: 24 October 2014 15:49:34 GMT+8 > To: "pkg-mysql-maint(a)lists.alioth.debian.org" <pkg-mysql-maint(a)lists.alioth.debian.org>, "Honza Horak" <hhorak(a)redhat.com>, "Roman Drahtmueller" <draht(a)suse.de> > > Hi package maintainers, > > We have a new plugin in MySQL 5.7 that makes it possible to have > accounts that can't log in: > > CREATE USER foo@localhost IDENTIFIED WITH 'mysql_no_login'; > > The mysql_no_login plugin simply denies all login attempts. This is > useful for users that are created, e.g., to serve as proxy users, or > as owners of stored programs/functions, views or events. > > This new plugin doesn't fix known security defects in the server, but > does provide new and better means to harden security. Best practices > for security include application of least-required privileges, and in > some cases, that means no client connections for privileged > accounts. This new plugin provides means to implement such > restrictions in a standard way. > > Because of the security benefits, we'd like to discuss backporting it > to 5.6. Like you, we don't like big changes to GA releases, but this > time we think it has a good use case, it's safe and has a very low > risk of regressions: > > - Since this is a plugin, it doesn't touch server code > - All new code is in a plugin that must be enabled explicitly by the > DBA > - The code itself is very simple. It's only one line of "real" code > (unconditionally return authentication failure), plus necessary > plugin plumbing to fill out the plugin API. > > If we backport this to 5.6, there are multiple ways to avoid it: > > - Apply a patch from us to remove the plugin > - Don't build it > - Build it, but don't ship it > - Build and ship it, but don't use it (in any case, the DBA has to > enable it and alter the user accounts to use it) > > So what do you think about backporting this? The only thing you'll > notice is one more file in the plugins directory. > > Regards, > > Norvald > > _______________________________________________ > pkg-mysql-maint mailing list > pkg-mysql-maint(a)lists.alioth.debian.org > http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-mysql-maint -- Colin Charles, Chief Evangelist, MariaDB Corporation blog: http://bytebot.net/blog/| t: +6-012-204-3201 | Skype: colincharles

2 1

Re: [Maria-developers] [Commits] b28e6bf: MDEV-6945 MDEV-6948 MDEV-6956 MDEV-6957 MDEV-6953 MDEV-6954 MDEV-6955
by Sergei Golubchik 29 Oct '14

29 Oct '14

Hi, Sanja! On Oct 29, sanja(a)mariadb.com wrote: > --- > mysql-test/r/set_statement.result | 14 ++++++++++++++ > mysql-test/r/set_statement_profiling.result | 4 ++++ > mysql-test/t/set_statement.test | 18 ++++++++++++++++++ > mysql-test/t/set_statement_profiling.test | 9 +++++++++ Did you put set_statement_debug.test in a separate commit? That's good. > sql/set_var.h | 4 +++- > sql/share/errmsg-utf8.txt | 2 ++ > sql/sql_parse.cc | 5 +++++ > sql/sys_vars.cc | 20 +++++++++++--------- > sql/sys_vars.h | 1 + > 9 files changed, 67 insertions(+), 10 deletions(-) > > diff --git a/sql/share/errmsg-utf8.txt b/sql/share/errmsg-utf8.txt > index bb42807..1633169 100644 > --- a/sql/share/errmsg-utf8.txt > +++ b/sql/share/errmsg-utf8.txt > @@ -7113,3 +7113,5 @@ ER_STATEMENT_TIMEOUT 70100 > eng "Query execution was interrupted (max_statement_time exceeded)" > ER_SUBQUERIES_NOT_SUPPORTED 42000 > eng "%s does not support subqueries or stored functions." > +ER_SET_STATEMENT_NOT_SUPPORTED 42000 > + eng "Variable '%s' is not supported in SET STATEMENT." Use the text eng "The system variable %.200s cannot be set in SET STATEMENT." > diff --git a/sql/sys_vars.cc b/sql/sys_vars.cc > index 6252e89..fd77e5f 100644 > --- a/sql/sys_vars.cc > +++ b/sql/sys_vars.cc > @@ -997,7 +997,7 @@ static bool check_master_connection(sys_var *self, THD *thd, set_var *var) > static Sys_var_session_lexstring Sys_default_master_connection( > "default_master_connection", > "Master connection to use for all slave variables and slave commands", > - SESSION_ONLY(default_master_connection), > + NO_SET_STMT SESSION_ONLY(default_master_connection), Why should this be prohibited? I'd rather have it supported. Use case: setting many connection variables for a single master, like in SET STATEMENT default_master_connection=foobar SET max_relay_log_size=10000000, sql_slave_skip_counter=5; Admittedly, this use case, while valid, isn't a strong argument at the moment - we don't have many per-master variables yet. When we'll have more, this will become more important. So, if fixing default_master_connection is not easy - keep it NO_SET_STMT, it can be enabled later. Why, exactly, does it fail? > NO_CMD_LINE, IN_SYSTEM_CHARSET, > DEFAULT(""), MAX_CONNECTION_NAME, ON_CHECK(check_master_connection)); > #endif > @@ -3527,12 +3528,12 @@ static Sys_var_bit Sys_unique_checks( > #ifdef ENABLED_PROFILING > static Sys_var_bit Sys_profiling( > "profiling", "profiling", > - SESSION_VAR(option_bits), NO_CMD_LINE, OPTION_PROFILING, > + NO_SET_STMT SESSION_VAR(option_bits), NO_CMD_LINE, OPTION_PROFILING, Why should this be prohibited? I'd rather have it supported. Use case: one usually needs to profile a single statement, not everything. > DEFAULT(FALSE)); > > static Sys_var_ulong Sys_profiling_history_size( > "profiling_history_size", "Limit of query profiling memory", > - SESSION_VAR(profiling_history_size), CMD_LINE(REQUIRED_ARG), > + NO_SET_STMT SESSION_VAR(profiling_history_size), CMD_LINE(REQUIRED_ARG), > VALID_RANGE(0, 100), DEFAULT(15), BLOCK_SIZE(1)); > #endif > > @@ -3566,7 +3567,8 @@ static bool check_skip_replication(sys_var *self, THD *thd, set_var *var) > > static Sys_var_bit Sys_skip_replication( > "skip_replication", "skip_replication", > - SESSION_ONLY(option_bits), NO_CMD_LINE, OPTION_SKIP_REPLICATION, > + NO_SET_STMT SESSION_ONLY(option_bits), > + NO_CMD_LINE, OPTION_SKIP_REPLICATION, Why should this be prohibited? I'd rather have it supported. Use case: skip replication of a single statement. A generic version of FLUSH NO_WRITE_TO_BINLOG TABLE REPAIR NO_WRITE_TO_BINLOG TABLE ANALYZE NO_WRITE_TO_BINLOG TABLE OPTIMIZE NO_WRITE_TO_BINLOG TABLE and all ALTER TABLE ... PARTITION variants: ALTER TABLE ... REBUILD PARTITION NO_WRITE_TO_BINLOG ALTER TABLE ... OPTIMIZE PARTITION NO_WRITE_TO_BINLOG ALTER TABLE ... ANALYZE PARTITION NO_WRITE_TO_BINLOG ALTER TABLE ... REPAIR PARTITION NO_WRITE_TO_BINLOG ALTER TABLE ... COALESCE PARTITION NO_WRITE_TO_BINLOG ALTER TABLE ... ADD PARTITION NO_WRITE_TO_BINLOG ALTER TABLE ... REORGANIZE PARTITION NO_WRITE_TO_BINLOG > DEFAULT(FALSE), NO_MUTEX_GUARD, NOT_IN_BINLOG, > ON_CHECK(check_skip_replication)); > > @@ -3688,7 +3690,7 @@ static ulonglong read_rand_seed(THD *thd) > static Sys_var_session_special Sys_rand_seed1( > "rand_seed1", "Sets the internal state of the RAND() " > "generator for replication purposes", > - sys_var::ONLY_SESSION, NO_CMD_LINE, > + NO_SET_STMT sys_var::ONLY_SESSION, NO_CMD_LINE, > VALID_RANGE(0, ULONG_MAX), BLOCK_SIZE(1), > NO_MUTEX_GUARD, IN_BINLOG, ON_CHECK(0), > ON_UPDATE(update_rand_seed1), ON_READ(read_rand_seed)); > @@ -3706,7 +3708,7 @@ static bool update_rand_seed2(THD *thd, set_var *var) > static Sys_var_session_special Sys_rand_seed2( > "rand_seed2", "Sets the internal state of the RAND() " > "generator for replication purposes", > - sys_var::ONLY_SESSION, NO_CMD_LINE, > + NO_SET_STMT sys_var::ONLY_SESSION, NO_CMD_LINE, > VALID_RANGE(0, ULONG_MAX), BLOCK_SIZE(1), > NO_MUTEX_GUARD, IN_BINLOG, ON_CHECK(0), > ON_UPDATE(update_rand_seed2), ON_READ(read_rand_seed)); I'd rather fix rand_seed1/rand_seed2 variables not to be special anymore. Like, move them to SV structure, remove special behavior and so on. Regards, Sergei

1 0

Re: [Maria-developers] [Commits] b18ddd2: Added 'mariadb-galera-server' metapackage to install all
by Sergei Golubchik 28 Oct '14

28 Oct '14

Hi, Nirbhay! On Oct 23, nirbhay(a)skysql.com wrote: > > Added 'mariadb-galera-server' metapackage to install all > necessary Galera-related dependencies. > > --- > debian/dist/Debian/control | 13 ++++++++++--- > debian/dist/Ubuntu/control | 13 ++++++++++--- > 2 files changed, 20 insertions(+), 6 deletions(-) > > diff --git a/debian/dist/Debian/control b/debian/dist/Debian/control > index e68f027..bc48dbc 100644 > --- a/debian/dist/Debian/control > +++ b/debian/dist/Debian/control > @@ -194,9 +194,7 @@ Recommends: libhtml-template-perl > Pre-Depends: mariadb-common, adduser (>= 3.40), debconf > Depends: mariadb-client-10.1 (>= ${source:Version}), libdbi-perl, > perl (>= 5.6), ${shlibs:Depends}, ${misc:Depends}, psmisc, passwd, > - lsb-base (>= 3.0-10), mariadb-server-core-10.1 (>= ${binary:Version}), > - galera (>=25.2), rsync, lsof, socat | netcat, grep, gawk, iproute, > - coreutils, findutils > + lsb-base (>= 3.0-10), mariadb-server-core-10.1 (>= ${binary:Version}) > Provides: mariadb-server, mysql-server, virtual-mysql-server > Conflicts: mariadb-server (<< ${source:Version}), mysql-server (<< ${source:Version}), > mysql-server-4.1, mysql-server-5.0, mysql-server-5.1, mysql-server-5.5, Looks ok. Few comments: 1. What about RPM? 2. Did you compare with, say, non-galera 10.0 - just to make sure all dependencies you removed are, indeed, only for galera and the server doesn't need any of that? 3. I'd also add /etc/mysql/conf.d/galera.cnf (/etc/my.cnf.d/galera.cnf in rpms) that enables wsrep, and may be sets other reasonable defaults for galera. Regards, Sergei

1 0

[Maria-developers] DisconnectEx() problem
by Delveri chick 28 Oct '14

28 Oct '14

Hi i was tracing why mysql_socket_shutdown(...) won't call DisconnectEx(...) as reported in the TODO list and realised DisconnectEx(...) cannot be called in include/mysql/psi/mysql_socket.h. Possible solution would be to add it here or include the *.h where it it defined. Where is it defined please? Regards Delveri

2 1

Re: [Maria-developers] [Commits] 18a7470: MDEV-6939 : Dots in file names of configuration files
by Sergei Golubchik 28 Oct '14

28 Oct '14

Hi, Nirbhay! 1. Why 10.1? 1.0 would be fine too, may be even 5.5 2. In cases like this I usually make two commits, one for a cleanup and one for a bug fix. Anyway, the fix is ok to push, of course. On Oct 24, nirbhay(a)mariadb.com wrote: > revision-id: 18a7470a563c30bffbfe787e4e5406b687597a44 > parent(s): cc656e4d8538326d85e84adbe48d0a3739d55674 > committer: Nirbhay Choubey > branch nick: server > timestamp: 2014-10-24 16:34:54 -0400 > message: > > MDEV-6939 : Dots in file names of configuration files > > Use fn_ext2() to get the file extension from last occurrence > of FN_EXTCHAR ('.') instead. Also made some cosmetic changes > in mysys/mf_fn_ext.c. > > --- > mysys/mf_fn_ext.c | 11 ++++++----- > mysys/my_default.c | 2 +- > 2 files changed, 7 insertions(+), 6 deletions(-) > Regards, Sergei

1 0

[Maria-developers] DATE and GROUP_MIN_MAX optimization
by Alexander Barkov 28 Oct '14

28 Oct '14

Hello Sergey, I'm working on pluggable data types and moving some pieces of the code into new classes for data type handlers. I noticed that GROUP_MIN_MAX optimization works fine with a TIME column with temporal and string literals but does not work for an integer literal: # This script creates demo data: DROP TABLE IF EXISTS t1; CREATE TABLE t1 (id INT, b TIME, KEY(id,b)) ENGINE=MyISAM; INSERT INTO t1 VALUES (1,10),(1,11),(1,12),(1,13),(1,14),(1,15),(1,16),(2,10),(2,11),(2,12),(2,13),(2,14),(2,15),(2,16); SELECT * FROM t1; # Now three EXPLAIN queries: explain SELECT id, MIN(b),MAX(b) FROM t1 WHERE b<TIME'10:10:10' GROUP BY id; explain SELECT id, MIN(b),MAX(b) FROM t1 WHERE b<'10:10:10' GROUP BY id; explain SELECT id, MIN(b),MAX(b) FROM t1 WHERE b<10 GROUP BY id; The first and the second EXPLAIN queries tell that 8 rows are to be examined; tracing the code in debugger shows that group_min_max optimization is active: > mysql> explain SELECT id, MIN(b),MAX(b) FROM t1 WHERE b<TIME'10:10:10' GROUP BY id; > +------+-------------+-------+-------+---------------+------+---------+------+------+---------------------------------------+ > | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | > +------+-------------+-------+-------+---------------+------+---------+------+------+---------------------------------------+ > | 1 | SIMPLE | t1 | range | NULL | id | 9 | NULL | 8 | Using where; Using index for group-by | > +------+-------------+-------+-------+---------------+------+---------+------+------+---------------------------------------+ > 1 row in set (0.00 sec) > > mysql> explain SELECT id, MIN(b),MAX(b) FROM t1 WHERE b<'10:10:10' GROUP BY id; > +------+-------------+-------+-------+---------------+------+---------+------+------+---------------------------------------+ > | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | > +------+-------------+-------+-------+---------------+------+---------+------+------+---------------------------------------+ > | 1 | SIMPLE | t1 | range | NULL | id | 9 | NULL | 8 | Using where; Using index for group-by | > +------+-------------+-------+-------+---------------+------+---------+------+------+---------------------------------------+ > 1 row in set (0.00 sec) > The third EXPLAIN query tells that all 14 rows is to be examined, and tracing in debugging shows that group_min_max is not enabled: > mysql> explain SELECT id, MIN(b),MAX(b) FROM t1 WHERE b<10 GROUP BY id; > +------+-------------+-------+-------+---------------+------+---------+------+------+--------------------------+ > | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | > +------+-------------+-------+-------+---------------+------+---------+------+------+--------------------------+ > | 1 | SIMPLE | t1 | index | NULL | id | 9 | NULL | 14 | Using where; Using index | > +------+-------------+-------+-------+---------------+------+---------+------+------+--------------------------+ > 1 row in set (0.01 sec) Any reasons not to use group_min_max for TIME column in a combination with an integer literal? I have a feeling that it should work. What about decimal and real literals? Thanks.

1 0

[Maria-developers] Problem with KVM version in Buildbot?
by Kristian Nielsen 24 Oct '14

24 Oct '14

Hi Daniel, We have a problem with a build in Buildbot: https://internal.askmonty.org/buildbot/builders/kvm-rpm-centos7_0-x86_64/bu… + qemu-img create -b /kvm/vms/vm-centos7_0-x86_64-build.qcow2 -f qcow2 /dev/shm/vm-tmp-2302.qcow2 qemu-img: '' uses a qcow2 feature which is not supported by this qemu version: QCOW version 3 qemu-img: Could not open '/kvm/vms/vm-centos7_0-x86_64-build.qcow2' Failed to clone base image, aborting This failure happens on the build host `terrier'. I assume the problem is that the image was created on a machine with a newer KVM? The host terrier seems to have a rather old one: knielsen@terrier:~> kvm --version QEMU emulator version 0.13.0 (qemu-kvm-devel), Copyright (c) 2003-2008 Fabrice Bellard How do you think we should solve the problem? Should we upgrade terrier? Or should we alternatively make sure to create new images in a backward-compatible version (and re-create affected images) ? Also, another issue related to VM images. Our old images were created with rather small disk size. I noticed that some new images are created much larger, for example: $ qemu-img info /kvm/vms/vm-sid-amd64-build.qcow2 image: /kvm/vms/vm-sid-amd64-build.qcow2 file format: qcow2 virtual size: 40G (42949672960 bytes) disk size: 3.1G cluster_size: 65536 Format specific information: compat: 0.10 That is fine, I think, it is indeed annoying when we run out of space in a build VM. However, the reason I noticed this is because host bb02 ran out of space in /run/shm/ due to a 16 GB copy of this image. So when we use larger VM images, we cannot use the optimisation to have the temporary copies in ramdisk, lest we get disk full errors. If the virtual size is larger to (ram disk size) / (max_builds), then we need to disable using ramdisk for buildbot for that image. I will try to fix the config to not use ramdisk for existing images that have too large virtual size. But it is something to keep in mind for the future. - Kristian.

3 3

Re: [Maria-developers] SQL Accelerator FPGA/ASIC
by Nasser Ghoseiri 24 Oct '14

24 Oct '14

Dear Jan, Thank you for your response. I have background in SQL (MS, Sybase, MySQL, Postgre), so luckily I'm not a beginner in query writing, however, I have absolutely no background in how databases work on background (other than hash-table). The idea is not really SIMD here, but a specially designed hardware to perform database functions much faster than the conventional DISK<->RAM<->CACHE<->CPU model. What I'm struggling to figure is that the latency caused by a enterprise-grade SQL script (not very correct and advanced code, but not very low-quality either), is Memory-Related or Arithmetic related or both? Here is what can be made for example: 1) A card with 16 DDR3 Chips on board and a Central FPGA, allowing very fast access to data, and like 100 small CPUs integrated in the FPGA to process a query. (a query that was written to take advantage of these features however) 2) A card with XDR Memory chips and like 32GB of Flash Chips to achieve very high-speed storage, retrieval (SSD<->RAM) and an FPGA that has many cores inside. A query is executed by FPGA pulling large chunk of data from Flashes, putting them in XDR RAM chips and processes them in a parallel manner 3) A card with XDR Memory chips and an FPGA on board, connected to a PCI Express 8x which receives all the data it needs from the host computer, and does the parallel processing inside itself, returning it back to the host. 4) A card with an FPGA inside, having 100 small CPUs and PCI Express to take commands from host computer and process in a parallel manner 5) A card with an FPGA, not having small processors inside, but special circuitry that does either lookup, sorting, etc in a very fast manner These are a few examples that I gave. The difference among them is the notion of having RAM, having a cold-storage, having an FPGA with many CPUs inside or an FPGA with special dedicated circuitry that performs special functions in parallel that is difficult for a CPU to handle (imaging adding 300 numbers to another 300 numbers in a single cycle, where a CPU with SIMD would take much longer, or maybe a string processor that processes many string columns with PATINDEX, etc in parallel where a CPU would be slow as it can't handle them in parallel). So what I'm researching is whether beneath the SQL pyramid, there is a memory intensive operation going on or logic-intensive. What to look for and what to aim for... Thanks in advance for your help, Nasser On Friday, October 24, 2014 7:49 PM, Jan Lindström <jan.lindstrom(a)mariadb.com> wrote: Hi, This idea of SIMD (single instruction multiple data) processing is not totally new one, similarly the idea to perform SQL-operations inside GPU or GPGA is not new. In traditional relational databases problematic is the fact that e.g. in your example TableX contains several columns, picking columns A and B from pages that reside first on disk, then on main-memory and finally on L1-L3 cache is not cheap, and then they are not on continuous memory. This is because page in cache would contain values for other columns that we not even need. In columnar database architecture this would be a lot easier, you just feed column containing values for A and B directly to SIMD operation and every page in main memory would contain a lot more values to process compared to traditional relational database where page would contain also values for columns that we really do not even need to process the result set of query. Anyway, I find the proposal interesting and challenging. R: Jan Lindström On Thu, Oct 23, 2014 at 3:39 PM, Nasser Ghoseiri <cdmcsd(a)yahoo.com> wrote: Dear Serg, > > >Following our little chat in IRC, I'm writing this email to explain in more detail what the idea is. My name is Nasser GHOSEIR (Founder, CTO of Butterfly labs), which I must note that this project is not from Butterfly Labs, >but will be a new company in Europe. > > >Our idea is to find a way to accelerate SQL query processing by either: > > >1) Creating an FPGA solution (Which will later evolve into ASIC), that has like 400 processors in itself, allow distribute calculation of some kind >2) Creating an FPGA/ASIC solution that performs large number of unrelated tasks (such as addition, multiplication, etc) in parallel >3) Creating an FPGA/ASIC solution that allows very high-speed access to data with some PRE-PROCESSING involved to accelerate the calculation >4) Creating a very high-speed storage solution, but no pre-processing. > > >An example to give is for 200,000 rows, imagine: "SELECT A+B, C FROM TableX WHERE C > 0". If there are 200,000 records, the processor has to perform 200,000 additions (A+B). >A CPU will handle these additions one-by-one (or maybe few-by-few if SSE, etc is used). However, an FPGA/ASIC solution can perform 1000 additions in a single-cycle. This results in acceleration. > > >Now, to what extend this can be effective, or what other solutions (maybe string processing?) can be implemented to accelerate the SQL processing is a question to me right now. But we do have experience in >making extremely fast processors (our Butterfly Labs Monarch chip performs 400 billion double-SHA256 hashes in 1 second, around 20,000 times faster than the best intel XEON processor can do only twenty >million hashes per second). > > >The project is in brain-storming phase, and we are aware that this solution will be useful to large enterprises, or companies that have to deal with 200,000 rows in a single "SELECT" query. The idea is to integrate >some features in hardware, and then re-write a portion of MariaDB to take advantage of the new resources. Also, it is possible that users/companies will need to re-write their query to make it compliant. > > >You can reach my by mail at cdmcsd(a)yahoo.com or by phone at +33 6 72 17 26 19 (France). > > > > > > >Best Regards, >Nasser GHOSEIRI > > > > > > > > >_______________________________________________ >Mailing list: https://launchpad.net/~maria-developers >Post to : maria-developers(a)lists.launchpad.net >Unsubscribe : https://launchpad.net/~maria-developers >More help : https://help.launchpad.net/ListHelp > >

1 0

[Maria-developers] Please review: MDEV-5528 Command line variable to choose MariaDB-5.3 vs MySQL-5.6 temporal data formats
by Alexander Barkov 24 Oct '14

24 Oct '14

Hi, Please review my patch for MDEV-5528. It also fixes a bug: MDEV-6649 Different warnings for TIME and TIME(N) when @@old_mode=zero_date_time_cast Thanks.

2 8

[Maria-developers] doubt about mariadb dialog plugin at server and client side
by Roberto Spadim 23 Oct '14

23 Oct '14

Hi guys I'm with a doubt at dialog plugin I created a user at mariadb server: ---SQL INSTALL PLUGIN three_attempts SONAME 'dialog_examples.so'; CREATE USER test_dialog IDENTIFIED VIA three_attempts USING 'SECRET'; FLUSH PRIVILEGES; --- now i'm trying to connect as user test_dialog: ---SHELL $mysql *--plugin-dir=/home/rspadim/mariadb/mariadb-10.0.14/plugin/auth_dialog* --user=test_dialog --host=spd1.spadim.com.br --password Enter password: (i just hit enter) [mariadb] Password, please: (enter again) [mariadb] Password, please: (SECRET enter) Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 49243 Server version: 5.5.5-10.0.14-MariaDB-log Source distribution Copyright (c) 2000, 2012, Oracle, Monty Program Ab and others. Reading history-file /home/beto/.mysql_history Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> quit Writing history-file /home/beto/.mysql_history --- ---SHELL $ls /home/rspadim/mariadb/mariadb-10.0.14/plugin/auth_dialog CMakeFiles cmake_install.cmake CMakeLists.txt CTestTestfile.cmake dialog.c *dialog.so* Makefile --- ---MYSQL.USER table || *Host* || *User* || *Password* || *Select_priv* || *Insert_priv* || *Update_priv* || *Delete_priv* || *Create_priv* || *Drop_priv* || *Reload_priv* || *Shutdown_priv* || *Process_priv* || *File_priv* || *Grant_priv* || *References_priv* || *Index_priv* || *Alter_priv* || *Show_db_priv* || *Super_priv* || *Create_tmp_table_priv* || *Lock_tables_priv* || *Execute_priv* || *Repl_slave_priv* || *Repl_client_priv* || *Create_view_priv* || *Show_view_priv* || *Create_routine_priv* || *Alter_routine_priv* || *Create_user_priv* || *Event_priv* || *Trigger_priv* || *Create_tablespace_priv* || *ssl_type* || *ssl_cipher* || *x509_issuer* || *x509_subject* || *max_questions* || *max_updates* || *max_connections* || *max_user_connections* || *plugin* || *auth_string* || || % || test_dialog || >>>CHECK NO PASSWORD!<<< || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || N || || || || || 0 || 0 || 0 || 0 || three_attempts || SECRET || --- the doubts: 1) why it accept my login with password? i *don't* have a password at mysql.user schema 2) why i didn't received a dialog like in heidisql? 3) there's a auth trace or auth verbose option at client side? or any other command to check if i loaded the dialog plugin? -- Roberto Spadim

2 3

[Maria-developers] SQL Accelerator FPGA/ASIC
by Nasser Ghoseiri 23 Oct '14

23 Oct '14

Dear Serg, Following our little chat in IRC, I'm writing this email to explain in more detail what the idea is. My name is Nasser GHOSEIR (Founder, CTO of Butterfly labs), which I must note that this project is not from Butterfly Labs, but will be a new company in Europe. Our idea is to find a way to accelerate SQL query processing by either: 1) Creating an FPGA solution (Which will later evolve into ASIC), that has like 400 processors in itself, allow distribute calculation of some kind 2) Creating an FPGA/ASIC solution that performs large number of unrelated tasks (such as addition, multiplication, etc) in parallel 3) Creating an FPGA/ASIC solution that allows very high-speed access to data with some PRE-PROCESSING involved to accelerate the calculation 4) Creating a very high-speed storage solution, but no pre-processing. An example to give is for 200,000 rows, imagine: "SELECT A+B, C FROM TableX WHERE C > 0". If there are 200,000 records, the processor has to perform 200,000 additions (A+B). A CPU will handle these additions one-by-one (or maybe few-by-few if SSE, etc is used). However, an FPGA/ASIC solution can perform 1000 additions in a single-cycle. This results in acceleration. Now, to what extend this can be effective, or what other solutions (maybe string processing?) can be implemented to accelerate the SQL processing is a question to me right now. But we do have experience in making extremely fast processors (our Butterfly Labs Monarch chip performs 400 billion double-SHA256 hashes in 1 second, around 20,000 times faster than the best intel XEON processor can do only twenty million hashes per second). The project is in brain-storming phase, and we are aware that this solution will be useful to large enterprises, or companies that have to deal with 200,000 rows in a single "SELECT" query. The idea is to integrate some features in hardware, and then re-write a portion of MariaDB to take advantage of the new resources. Also, it is possible that users/companies will need to re-write their query to make it compliant. You can reach my by mail at cdmcsd(a)yahoo.com or by phone at +33 6 72 17 26 19 (France). Best Regards, Nasser GHOSEIRI

1 0

[Maria-developers] COMPOUND STATEMENT fails on Windows.
by Peter Laursen 22 Oct '14

22 Oct '14

I tried to comment on this Blog https://blog.mariadb.org/mariadb-10-1-1-compound-statements/, but it won't let me use my (corporate) gmail address for authencticaton. So I send this mail instead: I tried the example provided on Windows (both server and client are Win7 64 bit) using the command line client shipped with MariaDB 10.1: Enter password: ******** Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 5 Server version: 10.1.1-MariaDB mariadb.org binary distribution Copyright (c) 2000, 2014, Oracle, SkySQL Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> use test; Database changed MariaDB [test]> IF @have_csv = 'YES' THEN -> CREATE TABLE IF NOT EXISTS general_log ( -> event_time TIMESTAMP(6) NOT NULL, -> user_host MEDIUMTEXT NOT NULL, -> thread_id BIGINT(21) UNSIGNED NOT NULL, -> server_id INTEGER UNSIGNED NOT NULL, -> command_type VARCHAR(64) NOT NULL, -> argument MEDIUMTEXT NOT NULL -> ) engine=CSV CHARACTER SET utf8 comment="General log"; ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' a t line 9 MariaDB [test]> END IF; Obviously the SEMICOLON after comment="General log" disturbs. But this is no better: MariaDB [test]> IF @have_csv = 'YES' THEN -> CREATE TABLE IF NOT EXISTS general_log ( -> event_time TIMESTAMP(6) NOT NULL, -> user_host MEDIUMTEXT NOT NULL, -> thread_id BIGINT(21) UNSIGNED NOT NULL, -> server_id INTEGER UNSIGNED NOT NULL, -> command_type VARCHAR(64) NOT NULL, -> argument MEDIUMTEXT NOT NULL -> ) engine=CSV CHARACTER SET utf8 comment="General log" -> END IF; ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'END IF' at line 10 MariaDB [test]> What is the problem here? Is it someWindows-specific bug/issue? BTW it is the same in SQLyog (compiled with MariaDB C-API) as in commandline. -- Peter -- Webyog

2 2

Re: [Maria-developers] [Commits] 28fcccc: MDEV-5231: Per query variables from Percona Server (rewritten)
by Sergei Golubchik 22 Oct '14

22 Oct '14

Hi, Sanja! On Oct 13, sanja(a)mariadb.com wrote: > revision-id: 28fccccad32fa8a47dc572b968613e18ff51b634 > parent(s): fec5ab5a56cb9a45c621207620cc85079cddf537 > committer: Oleksandr Byelkin > branch nick: work-maria-10.1-sysvar2 > timestamp: 2014-10-13 22:35:40 +0200 > message: > > MDEV-5231: Per query variables from Percona Server (rewritten) This looks a lot better that the previous patch. Thanks! See my comments below. Some parts of the patch are not used (left-overs from the old implementation?). Some others, I think, are not needed. And some of the old tests don't make much sense, your new tests made them redundant and useless. A couple of comments after reading my previous review: * please add a test for any sysvar with non-trivial implementations of sys_var::session_update(), that is, for a sysvar where old approach doesn't work. Unless, of course, your timestamp tests are exactly that :) * add tests with @@default_engine. For example: SET @@default_engine=MyISAM; SET STATEMENT @@default_engine=MEMORY CREATE TABLE t1 (a int); SHOW CREATE TABLE t1; -- verify the engine SET STATEMENT @@default_engine=MyISAM CREATE TABLE t2 (a int); etc. See other comments below: > diff --git a/mysql-test/r/set_statement.result b/mysql-test/r/set_statement.result > new file mode 100644 > index 0000000..fed937e > --- /dev/null > +++ b/mysql-test/r/set_statement.result > @@ -0,0 +1,941 @@ > +'# SET STATEMENT ..... FOR .... TEST' > +DROP TABLE IF EXISTS t1; > +DROP FUNCTION IF EXISTS myProc; > +DROP PROCEDURE IF EXISTS p1; > +DROP PROCEDURE IF EXISTS p2; > +DROP PROCEDURE IF EXISTS p3; > +DROP PROCEDURE IF EXISTS p4; > +DROP PROCEDURE IF EXISTS p5; > +DROP TABLE IF EXISTS STATEMENT; > +'# Setup database' > +CREATE TABLE t1 (v1 INT, v2 INT); > +INSERT INTO t1 VALUES (1,2); > +INSERT INTO t1 VALUES (3,4); > +'' > +'#------------------ STATEMENT Test 1 -----------------------#' > +'# Initialize variables to known setting' > +SET SESSION sort_buffer_size=100000; > +'' > +'# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +SET STATEMENT sort_buffer_size=150000 FOR SELECT *,@@sort_buffer_size FROM t1; > +v1 v2 @@sort_buffer_size > +1 2 150000 > +3 4 150000 > +'' > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +'' > +'#------------------ STATEMENT Test 2 -----------------------#' > +'# Initialize variables to known setting' > +SET SESSION binlog_format=mixed; > +SET SESSION sort_buffer_size=100000; > +'# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +SHOW SESSION VARIABLES LIKE 'binlog_format'; > +Variable_name Value > +binlog_format MIXED > +SET STATEMENT sort_buffer_size=150000, binlog_format=row > +FOR SELECT *,@@sort_buffer_size,@@binlog_format FROM t1; > +v1 v2 @@sort_buffer_size @@binlog_format > +1 2 150000 ROW > +3 4 150000 ROW > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +SHOW SESSION VARIABLES LIKE 'binlog_format'; > +Variable_name Value > +binlog_format MIXED > +'' > +'#------------------ STATEMENT Test 3 -----------------------#' > +'# set initial variable value, make prepared statement > +SET SESSION binlog_format=row; > +PREPARE stmt1 FROM 'SET STATEMENT binlog_format=row FOR SELECT *,@@binlog_format FROM t1'; > +'' > +'# Change variable setting' > +SET SESSION binlog_format=mixed; > +'' > +'# Pre-STATEMENT variable value' > +'' > +SHOW SESSION VARIABLES LIKE 'binlog_format'; > +Variable_name Value > +binlog_format MIXED > +'' > +EXECUTE stmt1; > +v1 v2 @@binlog_format > +1 2 ROW > +3 4 ROW > +'' > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'binlog_format'; > +Variable_name Value > +binlog_format MIXED > +'' > +DEALLOCATE PREPARE stmt1; > +'#------------------ STATEMENT Test 4 -----------------------#' > +'# set initial variable value, make prepared statement > +SET SESSION myisam_sort_buffer_size=500000, myisam_repair_threads=1; > +'' > +'# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'myisam_sort_buffer_size'; > +Variable_name Value > +myisam_sort_buffer_size 500000 > +SHOW SESSION VARIABLES LIKE 'myisam_repair_threads'; > +Variable_name Value > +myisam_repair_threads 1 > +'' > +SET STATEMENT myisam_sort_buffer_size=800000, > +myisam_repair_threads=2 FOR OPTIMIZE TABLE t1; 1. this is not "make prepared statement" as the comment says 2. what does it actually test? you don't verify whether new values had any effect. > +Table Op Msg_type Msg_text > +test.t1 optimize status OK > +'' > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'myisam_sort_buffer_size'; > +Variable_name Value > +myisam_sort_buffer_size 500000 > +SHOW SESSION VARIABLES LIKE 'myisam_repair_threads'; > +Variable_name Value > +myisam_repair_threads 1 > +'' > +'#------------------ STATEMENT Test 5 -----------------------#' > +'# Initialize variables to known setting' > +SET SESSION sort_buffer_size=100000; > +'' > +'# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +'' > +SET STATEMENT sort_buffer_size=150000 FOR SELECT * FROM t2; > +ERROR 42S02: Table 'test.t2' doesn't exist > +'' > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 150000 Looks wrong. The value is not restored in case of an error. > +'' > +'#------------------ STATEMENT Test 6 -----------------------#' > +'# Initialize variables to known setting' > +SET SESSION keep_files_on_create=ON; > +'' > +'# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'keep_files_on_create'; > +Variable_name Value > +keep_files_on_create ON > +'' > +SET STATEMENT keep_files_on_create=OFF FOR SELECT *,@@keep_files_on_create FROM t1; > +v1 v2 @@keep_files_on_create > +1 2 0 > +3 4 0 How is this test different from other tests above? > +'' > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'keep_files_on_create'; > +Variable_name Value > +keep_files_on_create ON > +'' > +'#------------------ STATEMENT Test 7 -----------------------#' > +'# Initialize variables to known setting' > +SET SESSION max_join_size=2222220000000; > +'' > +'# Pre-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'max_join_size'; > +Variable_name Value > +max_join_size 2222220000000 > +'' > +SET STATEMENT max_join_size=1000000000000 FOR SELECT *,@@max_join_size FROM t1; > +v1 v2 @@max_join_size > +1 2 1000000000000 > +3 4 1000000000000 Same question - how is this test different from other tests above? > +'' > +'# Post-STATEMENT variable value' > +SHOW SESSION VARIABLES LIKE 'max_join_size'; > +Variable_name Value > +max_join_size 2222220000000 > +'' > +'#------------------Test 8-----------------------#' > +'# Initialize test variables' > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size = 200000, > +max_join_size=2222220000000, > +keep_files_on_create=ON; > +'' > +'# LONG ' > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 200000 > +SET STATEMENT sort_buffer_size = 100000 > +FOR SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 100000 > +SHOW SESSION VARIABLES LIKE 'sort_buffer_size'; > +Variable_name Value > +sort_buffer_size 200000 > +'' > +'# MY_BOOL ' > +SHOW SESSION VARIABLES LIKE 'keep_files_on_create'; > +Variable_name Value > +keep_files_on_create ON > +SET STATEMENT keep_files_on_create=OFF > +FOR SHOW SESSION VARIABLES LIKE 'keep_files_on_create'; > +Variable_name Value > +keep_files_on_create OFF > +SHOW SESSION VARIABLES LIKE 'keep_files_on_create'; > +Variable_name Value > +keep_files_on_create ON > +'' > +'# INT/LONG ' > +SHOW SESSION VARIABLES LIKE 'myisam_repair_threads'; > +Variable_name Value > +myisam_repair_threads 1 > +SET STATEMENT myisam_repair_threads=2 > +FOR SHOW SESSION VARIABLES LIKE 'myisam_repair_threads'; > +Variable_name Value > +myisam_repair_threads 2 > +SHOW SESSION VARIABLES LIKE 'myisam_repair_threads'; > +Variable_name Value > +myisam_repair_threads 1 > +'' > +'# ULONGLONG ' > +SHOW SESSION VARIABLES LIKE 'max_join_size'; > +Variable_name Value > +max_join_size 2222220000000 > +SET STATEMENT max_join_size=2000000000000 > +FOR SHOW SESSION VARIABLES LIKE 'max_join_size'; > +Variable_name Value > +max_join_size 2000000000000 > +SHOW SESSION VARIABLES LIKE 'max_join_size'; > +Variable_name Value > +max_join_size 2222220000000 > +'' > +'#------------------Test 9-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +CREATE FUNCTION myProc (cost DECIMAL(10,2)) > +RETURNS DECIMAL(10,2) > +SQL SECURITY DEFINER > +tax: BEGIN > +DECLARE order_tax DECIMAL(10,2); > +SET order_tax = cost * .05; > +RETURN order_tax; > +END| > +'' > +'# During Execution values > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +SELECT myProc(123.45); > +myProc(123.45) > +6.17 How does this verify that SET STATEMENT had any effect? > +'' > +'# Post-STATEMENT No 1 variable value Pre-STATEMENT for No 2' > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +DROP FUNCTION myProc; How does this verify that SET STATEMENT had any effect? > +'' > +'# Post-STATEMENT No 2 variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'#------------------Test 10-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +PREPARE stmt2 > +FROM 'SELECT * FROM t1'; > +'' > +'Test No 1 Post Value & Test 2 Pre values' > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +EXECUTE stmt2; > +v1 v2 > +1 2 > +3 4 How does this verify that SET STATEMENT had any effect? Add @@binlog_format and @@sort_buffer_size to the SELECT and use different values for @@sort_buffer_size in SET STATEMENT for PREPARE and EXECUTE. Ok, I see that you do it below. Then how is this test useful? > +'' > +'# Post-STATEMENT No 2 > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +DEALLOCATE PREPARE stmt2; > +'' > +'#------------------Test 11-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +PREPARE stmt1 FROM > +'SET STATEMENT binlog_format=row FOR SELECT *,@@binlog_format FROM t1'; please also add @@sort_buffer_size to this SELECT > +'' > +'Test No 1 Post Value & Test 2 Pre values' > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +EXECUTE stmt1; > +v1 v2 @@binlog_format > +1 2 ROW > +3 4 ROW > +'' > +'# Post-STATEMENT No 2 > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +'#------------------Test 12-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +CREATE PROCEDURE p1() BEGIN > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +END| > +'' > +'Test No 1 Post Value & Test 2 Pre values' > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR Use different values here, not same as in the previous SET STATEMENT > +CALL p1(); > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +400000 2 200000 ROW 0 4444440000000 > +'' > +'# Post-STATEMENT No 2 > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +'#------------------Test 13-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'' > +CREATE PROCEDURE p2() BEGIN > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=3, > +sort_buffer_size=300000, > +binlog_format=mixed, > +keep_files_on_create=OFF, > +max_join_size=3333330000000 FOR > +CALL p1(); > +END| > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +CALL p2(); > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +400000 3 300000 MIXED 0 3333330000000 > +'' > +'# Post-STATEMENT > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +'#------------------Test 14-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'' > +CREATE PROCEDURE p3() BEGIN > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +SET STATEMENT myisam_sort_buffer_size=320000, > +myisam_repair_threads=2, > +sort_buffer_size=220022, > +binlog_format=row, > +keep_files_on_create=ON, > +max_join_size=2222220000000 FOR > +CALL p2(); > +END| > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +CALL p3(); > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +400000 2 200000 ROW 0 4444440000000 > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +400000 3 300000 MIXED 0 3333330000000 > +'' > +'# Post-STATEMENT > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +'' > +'' > +'#------------------Test 15-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'' > +CREATE PROCEDURE p4() BEGIN > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +SET STATEMENT myisam_sort_buffer_size=320000, > +myisam_repair_threads=2, > +sort_buffer_size=220022, > +binlog_format=row, > +keep_files_on_create=ON, > +max_join_size=2222220000000 FOR > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +SET STATEMENT myisam_sort_buffer_size=320000, > +myisam_repair_threads=2, > +sort_buffer_size=220022, > +binlog_format=row, > +keep_files_on_create=ON, > +max_join_size=2222220000000 FOR > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +SET STATEMENT myisam_sort_buffer_size=320000, > +myisam_repair_threads=2, > +sort_buffer_size=220022, > +binlog_format=row, > +keep_files_on_create=ON, > +max_join_size=2222220000000 FOR > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +END| > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +CALL p4(); > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +400000 2 200000 ROW 0 4444440000000 > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +320000 2 220022 ROW 1 2222220000000 > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +320000 2 220022 ROW 1 2222220000000 > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +320000 2 220022 ROW 1 2222220000000 > +'' > +'# Post-STATEMENT > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +'#------------------Test 16-----------------------#' > +'' > +'# Pre-STATEMENT variable value > +SELECT @@sql_mode; > +@@sql_mode > + > +'' > +'' > +SET STATEMENT sql_mode='ansi' FOR PREPARE stmt FROM 'SELECT "t1".* FROM t1'; > +execute stmt; > +v1 v2 > +1 2 > +3 4 Ok, that's getting interesting. Now before deallocating, try to trigger automatic reprepare. e.g. ALTER TABLE t1 ADD COLUMN v3 int; and then execute it again. also do the same with SP. I mean, set statement sql_mode=ansi create procedure this procedure uses ansi quotes. verify that the correct sql mode (ansi) is recorded in mysql.proc table. > +deallocate prepare stmt; > +'' > +'# Post-STATEMENT > +SELECT @@sql_mode; > +@@sql_mode > + > +'' > +'' > +'#------------------Test 17-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=320000, > +myisam_repair_threads=2, > +sort_buffer_size=220022, > +binlog_format=row, > +keep_files_on_create=ON, > +max_join_size=2222220000000 > +FOR SET SESSION > +myisam_sort_buffer_size=260000, > +myisam_repair_threads=3, > +sort_buffer_size=230013, > +binlog_format=row, > +keep_files_on_create=ON, > +max_join_size=2323230000000; Add a couple of other variables to SET SESSION. The point - have some variables in SET SESSION that are *not* present in SET STATEMENT. Verify that their values were correctly set. Ok, I see that you do it below. Then how is this test useful? > +'' > +'# Post-STATEMENT > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +'#------------------Test 18-----------------------#' > +'# set initial variable values > +SET SESSION myisam_sort_buffer_size=500000, > +myisam_repair_threads=1, > +sort_buffer_size=100000, > +binlog_format=mixed, > +keep_files_on_create=ON, > +max_join_size=2222220000000; > +'' > +'# Pre-STATEMENT variable value > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +CREATE PROCEDURE p5() BEGIN > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +SET SESSION > +myisam_sort_buffer_size=260000, > +myisam_repair_threads=3, > +sort_buffer_size=230013, > +binlog_format=row, > +keep_files_on_create=ON, > +max_join_size=2323230000000; > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +END| How is this test different from other tests above? > +'' > +'' > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=2, > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +CALL p5(); > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +400000 2 200000 ROW 0 4444440000000 > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +260000 3 230013 ROW 1 2323230000000 > +'' > +'# Post-STATEMENT > +SELECT @@myisam_sort_buffer_size, > +@@myisam_repair_threads, > +@@sort_buffer_size, > +@@binlog_format, > +@@keep_files_on_create, > +@@max_join_size; > +@@myisam_sort_buffer_size @@myisam_repair_threads @@sort_buffer_size @@binlog_format @@keep_files_on_create @@max_join_size > +500000 1 100000 MIXED 1 2222220000000 > +'' > +'' > +'#------------------Test 19-----------------------#' > +SET STATEMENT max_error_count=100 FOR; > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' at line 1 > +SET STATEMENT max_error_count=100 INSERT t1 VALUES (1,2); > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'INSERT t1 VALUES (1,2)' at line 1 > +SET STATEMENT FOR INSERT INTO t1 VALUES (1,2); > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'FOR INSERT INTO t1 VALUES (1,2)' at line 1 > +SET max_error_count=100 FOR INSERT INTO t1 VALUES (1,2); > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'FOR INSERT INTO t1 VALUES (1,2)' at line 1 > +SET max_error_count=100 FOR INSERT INTO t1 VALUES (1,2); Same erroneous statement tested twice? > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'FOR INSERT INTO t1 VALUES (1,2)' at line 1 > +SET STATEMENT GLOBAL max_error_count=100 FOR INSERT INTO t1 VALUES (1,2); > +ERROR HY000: Unknown system variable 'GLOBAL' > +SET STATEMENT @@global.max_error_count=100 FOR INSERT INTO t1 VALUES (1,2); > +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '@@global.max_error_count=100 FOR INSERT INTO t1 VALUES (1,2)' at line 1 > +'' > +'' > +'#------------------Test 20-----------------------#' > +SET STATEMENT connect_timeout=100 FOR INSERT INTO t1 VALUES (1,2); > +ERROR HY000: Variable 'connect_timeout' is a GLOBAL variable and should be set with SET GLOBAL > +'' > +'' > +'#------------------Test 21-----------------------#' > +SELECT @@myisam_sort_buffer_size, @@sort_buffer_size; > +@@myisam_sort_buffer_size @@sort_buffer_size > +500000 100000 > +SET STATEMENT myisam_sort_buffer_size = 700000, sort_buffer_size = 3000000 > +FOR SET STATEMENT myisam_sort_buffer_size=200000 > +FOR SELECT @@myisam_sort_buffer_size, @@sort_buffer_size; > +@@myisam_sort_buffer_size @@sort_buffer_size > +200000 100000 > +SELECT @@myisam_sort_buffer_size, @@sort_buffer_size; > +@@myisam_sort_buffer_size @@sort_buffer_size > +500000 100000 > +'' > +'' > +'#------------------Test 22-----------------------#' > +CREATE TABLE STATEMENT(a INT); > +DROP TABLE STATEMENT; > +'' > +'# Cleanup' > +DROP TABLE t1; > +DROP PROCEDURE p1; > +DROP PROCEDURE p2; > +DROP PROCEDURE p3; > +DROP PROCEDURE p4; > +DROP PROCEDURE p5; > +CREATE TABLE t1 (v1 INT, v2 INT); > +PREPARE stmt > +FROM 'SELECT * FROM t1'; > +SET STATEMENT myisam_sort_buffer_size=400000, > +myisam_repair_threads=(select max(v1) from t1), > +sort_buffer_size=200000, > +binlog_format=row, > +keep_files_on_create=OFF, > +max_join_size=4444440000000 FOR > +EXECUTE stmt; > +ERROR HY000: SET STATEMENT does not support using tables in the assignment variables expressions Why do you need a prepared statement to test that error? > +deallocate prepare stmt; > +drop table t1; > +set session binlog_format=mixed; > +PREPARE stmt1 FROM 'SELECT @@binlog_format'; > +execute stmt1; > +@@binlog_format > +MIXED > +set statement binlog_format=row for execute stmt1; > +@@binlog_format > +ROW > +execute stmt1; > +@@binlog_format > +MIXED > +deallocate prepare stmt1; > +set statement binlog_format=row for PREPARE stmt1 FROM 'SELECT @@binlog_format'; > +execute stmt1; > +@@binlog_format > +MIXED > +execute stmt1; > +@@binlog_format > +MIXED > +deallocate prepare stmt1; > +PREPARE stmt1 FROM 'set statement binlog_format=row for SELECT @@binlog_format'; > +execute stmt1; > +@@binlog_format > +ROW > +execute stmt1; > +@@binlog_format > +ROW > +deallocate prepare stmt1; > +set session binlog_format=default; > +set session binlog_format=mixed; > +SET STATEMENT sort_buffer_size=150000 FOR set session binlog_format=row; > +SELECT @@binlog_format; > +@@binlog_format > +ROW those are good tests, and they make many of the tests above unnecessary > +#Correctly set timestamp > +set session timestamp=4646464; > +select @@timestamp != 4646464; > +@@timestamp != 4646464 > +0 > +select @@timestamp != 4646464; > +@@timestamp != 4646464 > +0 > +#Correctly returned normal behaviour > +set session timestamp=default; > +select @@timestamp != 4646464; > +@@timestamp != 4646464 > +1 > +select @@timestamp != 4646464; > +@@timestamp != 4646464 > +1 > +#here timestamp should be set only for the statement then restored default > +set statement timestamp=4646464 for select @@timestamp; > +@@timestamp > +4646464.000000 > +set @save_tm=@@timestamp; > +select @@timestamp != 4646464; > +@@timestamp != 4646464 > +1 > +select @@timestamp != 4646464; > +@@timestamp != 4646464 > +1 > +select @@timestamp != @save_tm; > +@@timestamp != @save_tm > +1 > diff --git a/sql/set_var.cc b/sql/set_var.cc > index bae6511..20e3040 100644 > --- a/sql/set_var.cc > +++ b/sql/set_var.cc > @@ -147,7 +147,7 @@ void sys_var_end() > flags(flags_arg), show_val_type(show_val_type_arg), > guard(lock), offset(off), on_check(on_check_func), on_update(on_update_func), > deprecation_substitute(substitute), > - is_os_charset(FALSE) > + is_os_charset(FALSE), default_val(TRUE) I've asked below why is that needed. > { > /* > There is a limitation in handle_options() related to short options: > @@ -695,7 +695,8 @@ int sql_set_variables(THD *thd, List<set_var_base> *var_list) > } > > err: > - free_underlaid_joins(thd, &thd->lex->select_lex); > + if (free) > + free_underlaid_joins(thd, &thd->lex->select_lex); Ok, so in normal SET you want to free_underlaid_joins, and in SET STATEMENT you don't. Right? > DBUG_RETURN(error); > } > > diff --git a/sql/item_func.cc b/sql/item_func.cc > index 2b89aa0..651e0b1 100644 > --- a/sql/item_func.cc > +++ b/sql/item_func.cc > @@ -5556,7 +5556,7 @@ longlong Item_func_get_user_var::val_int() > tmp_var_list.push_back(new set_var_user(new Item_func_set_user_var(name, > new Item_null()))); > /* Create the variable */ > - if (sql_set_variables(thd, &tmp_var_list)) > + if (sql_set_variables(thd, &tmp_var_list, true)) Make if 'false', not 'true' here. It doesn't make any practical difference (there are no joins, the variable is set to Item_null, explicitly), but makes the logic a bit clearer (standalone SET statement = free joins, a SET that is part of another statement = don't free joins). Alternatively (and better) approach would be to remove sql_set_variables() here and use the same code that Item_func_set_user_var does. And then sql_set_variables() will be only used in SET variants, where a sysvar can be used. Here sysvars aren't possible, so sql_set_variables() is misleading. > { > thd->lex= sav_lex; > goto err; > diff --git a/sql/set_var.h b/sql/set_var.h > index e48f394..9ee914b 100644 > --- a/sql/set_var.h > +++ b/sql/set_var.h > @@ -118,6 +119,9 @@ class sys_var > */ > bool set_default(THD *thd, set_var *var); > bool update(THD *thd, set_var *var); > + void stmt_update(THD *thd) { > + on_update && on_update(this, thd, OPT_SESSION); > + } Doesn't seem to be used anywhere in the patch > > String *val_str_nolock(String *str, THD *thd, const uchar *value); > longlong val_int(bool *is_null, THD *thd, enum_var_type type, const LEX_STRING *base); > diff --git a/sql/share/errmsg-utf8.txt b/sql/share/errmsg-utf8.txt > index 233bb83..4072af4 100644 > --- a/sql/share/errmsg-utf8.txt > +++ b/sql/share/errmsg-utf8.txt > @@ -7110,3 +7110,5 @@ ER_SLAVE_SKIP_NOT_IN_GTID > eng "When using GTID, @@sql_slave_skip_counter can not be used. Instead, setting @@gtid_slave_pos explicitly can be used to skip to after a given GTID position." > ER_STATEMENT_TIMEOUT 70100 > eng "Query execution was interrupted (max_statement_time exceeded)" > +ER_SET_STATEMENT_TABLES_IS_NOT_SUPPORTED > + eng "SET STATEMENT does not support using tables in the assignment variables expressions" Better write it as ER_SUBQUERIES_NOT_SUPPORTED eng "%s does not support subqueries or stored functions." and use it for SET STATEMENT: my_error(ER_SUBQUERIES_NOT_SUPPORTED, MYF(0), "SET STATEMENT"); and also use it for KILL: - my_error(ER_NOT_SUPPORTED_YET, MYF(0), "Usage of subqueries or stored " - "function calls as part of this statement"); + my_error(ER_SUBQUERIES_NOT_SUPPORTED, MYF(0), "KILL"); > diff --git a/sql/sql_class.h b/sql/sql_class.h > index d7bbfc3..0f6b39e 100644 > --- a/sql/sql_class.h > +++ b/sql/sql_class.h > @@ -3711,6 +3711,10 @@ class THD :public Statement, > void rgi_unlock_temporary_tables(); > bool rgi_have_temporary_tables(); > public: > + inline MEM_ROOT *get_execution_mem_root() > + { > + return &main_mem_root; > + } Doesn't seem to be used anywhere in this patch > /* > Flag, mutex and condition for a thread to wait for a signal from another > thread. > diff --git a/sql/sql_lex.h b/sql/sql_lex.h > index 5e9c7b9..03861d1 100644 > --- a/sql/sql_lex.h > +++ b/sql/sql_lex.h > @@ -2379,6 +2381,8 @@ struct LEX: public Query_tables_list > required a local context, the parser pops the top-most context. > */ > List<Name_resolution_context> context_stack; > + /* true if SET STATEMENT ... FOR ... statement is use, false otherwise */ > + bool set_statement; This doesn't seem to be needed, I've checked all places where set_statement is used below. > > SQL_I_List<ORDER> proc_list; > SQL_I_List<TABLE_LIST> auxiliary_table_list, save_list; > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index d8906b2..eff0e08 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -2637,6 +2637,98 @@ static bool do_execute_sp(THD *thd, sp_head *sp) > thd->get_binlog_format(&orig_binlog_format, > &orig_current_stmt_binlog_format); > > + if (lex->set_statement && !lex->stmt_var_list.is_empty()) Why do you check lex->set_statement? I'd think that if (!lex->stmt_var_list.is_empty()) is quite enough. > + { > + DBUG_PRINT("info", ("SET STATEMENT %d vars", lex->stmt_var_list.elements)); > + > + lex->old_var_list.empty(); > + List_iterator_fast<set_var_base> it(lex->stmt_var_list); > + set_var_base *var; > + while ((var=it++)) > + { > + DBUG_ASSERT(var->is_system()); > + set_var *o= NULL, *v= (set_var*)var; > + if (v->var->is_default()) > + o= new set_var(v->type, v->var, &v->base, NULL); Why do you treat default values differently? > + else > + { > + switch (v->var->option.var_type) > + { > + case GET_BOOL: > + case GET_INT: > + case GET_LONG: > + case GET_LL: > + { > + bool null_value; > + longlong val= v->var->val_int(&null_value, thd, v->type, &v->base); > + o= new set_var(v->type, v->var, &v->base, > + (null_value ? > + (Item *)new Item_null() : > + (Item *)new Item_int(val))); > + } > + break; > + case GET_UINT: > + case GET_ULONG: > + case GET_ULL: > + { > + bool null_value; > + ulonglong val= v->var->val_int(&null_value, thd, v->type, &v->base); > + o= new set_var(v->type, v->var, &v->base, > + (null_value ? > + (Item *)new Item_null() : > + (Item *)new Item_uint(val))); > + } > + break; > + case GET_DOUBLE: > + { > + bool null_value; > + double val= v->var->val_real(&null_value, thd, v->type, &v->base); > + o= new set_var(v->type, v->var, &v->base, > + (null_value ? > + (Item *)new Item_null() : > + (Item *)new Item_float(val, 1))); > + } > + break; > + default: > + case GET_NO_ARG: > + case GET_DISABLED: > + DBUG_ASSERT(0); > + case 0: > + case GET_FLAGSET: > + case GET_ASK_ADDR: > + case GET_TYPE_MASK: > + case GET_ENUM: > + case GET_SET: > + case GET_STR: > + case GET_STR_ALLOC: > + { > + char buff[STRING_BUFFER_USUAL_SIZE]; > + String tmp(buff, sizeof(buff), v->var->charset(thd)),*val; > + val= v->var->val_str(&tmp, thd, v->type, &v->base); > + if (val) > + { > + Item_string *str= new Item_string(v->var->charset(thd)); > + str->set_str_with_copy(val->ptr(), val->length()); > + o= new set_var(v->type, v->var, &v->base, str); > + } > + else > + o= new set_var(v->type, v->var, &v->base, new Item_null()); > + } > + break; > + } > + } > + DBUG_ASSERT(o); > + lex->old_var_list.push_back(o); > + } > + if (thd->is_error() || > + (res= sql_set_variables(thd, &lex->stmt_var_list, false))) > + { > + if (!thd->is_error()) > + my_error(ER_WRONG_ARGUMENTS, MYF(0), "SET"); > + goto error; > + } > + } > + > /* > Force statement logging for DDL commands to allow us to update > privilege, system or statistic tables directly without the updates > @@ -2812,6 +2904,12 @@ static bool do_execute_sp(THD *thd, sp_head *sp) > } > case SQLCOM_EXECUTE: > { > + /* > + Clean up free_list here from Items used in setting statement variables > + to present clear item list to Prepared_statement::execute_loop which > + need it empty because switch arena (and so item list) on statement arena. > + */ > + //free_items(thd->free_list); thd->free_list= NULL; Eh? Forgot to remove it? > mysql_sql_stmt_execute(thd); > break; > } > diff --git a/sql/sql_plugin.cc b/sql/sql_plugin.cc > index 9ae3d79..4b54989 100644 > --- a/sql/sql_plugin.cc > +++ b/sql/sql_plugin.cc > @@ -4078,3 +4078,48 @@ static void restore_ptr_backup(uint n, st_ptr_backup *backup) > (backup++)->restore(); > } > > + > +/** > + Create deep copy of system_variables instance. > +*/ > +struct system_variables * > +copy_system_variables(const struct system_variables *src) Doesn't seem to be used anywhere in the patch > +{ > + struct system_variables *dst; > + > + DBUG_ASSERT(src); > + > + dst= (struct system_variables *) > + sql_alloc(sizeof(struct system_variables)); > + if (!dst) > + return NULL; > + *dst= *src; > + > + if (dst->dynamic_variables_ptr) > + { > + if (!(dst->dynamic_variables_ptr= > + (char *)my_malloc(dst->dynamic_variables_size, MYF(MY_WME | MY_FAE)))) > + return NULL; > + memcpy(dst->dynamic_variables_ptr, > + src->dynamic_variables_ptr, > + src->dynamic_variables_size); > + } > + > + mysql_mutex_lock(&LOCK_plugin); > + dst->table_plugin= > + intern_plugin_lock(NULL, src->table_plugin); > + mysql_mutex_unlock(&LOCK_plugin); > + > + return dst; > +} > + > +void free_system_variables(struct system_variables *v) Doesn't seem to be used anywhere in the patch > +{ > + DBUG_ASSERT(v); > + > + mysql_mutex_lock(&LOCK_plugin); > + intern_plugin_unlock(NULL, v->table_plugin); > + mysql_mutex_unlock(&LOCK_plugin); > + > + my_free(v->dynamic_variables_ptr); > +} > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index e7fcdfb..d140597 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -14416,8 +14420,52 @@ set: > } > start_option_value_list > {} > + | SET STATEMENT_SYM > + { > + LEX *lex= Lex; > + mysql_init_select(lex); > + lex->sql_command= SQLCOM_SET_OPTION; > + /* Don't clear var_list in the case of recursive statement */ What's "recursive set statement"? SET STATEMENT a=1 FOR SET STATEMENT ? I'd simply return a syntax error here. Like MYSQL_YYABORT_UNLESS(lex->stmt_var_list.is_empty()); > + if (!lex->set_statement) > + { > + lex->var_list.empty(); > + lex->stmt_var_list.empty(); Why here, and not in lex_start() ? > + } > + lex->autocommit= 0; > + lex->set_statement= true; > + sp_head *sp= lex->sphead; > + if (sp && !sp->is_invoked()) > + { > + sp->m_param_begin= ((yychar == YYEMPTY) ? YYLIP->get_ptr() : YYLIP->get_tok_start()); > + sp->m_param_end= ((yychar == YYEMPTY) ? YYLIP->get_ptr() : YYLIP->get_tok_end()); what's that? > + } > + } > + set_stmt_option_value_following_option_type_list > + { > + LEX *lex= Lex; > + if (lex->query_tables) > + { > + my_error(ER_SET_STATEMENT_TABLES_IS_NOT_SUPPORTED, MYF(0)); why? what about stored functions that use tables internally? > + MYSQL_YYABORT; > + } > + //if (lex->set_statement && lex->stmt_var_list.is_empty()) > + { > + lex->stmt_var_list= lex->var_list; > + lex->var_list.empty(); > + } > + } > + FOR_SYM statement > + {} > ; > > +set_stmt_option_value_following_option_type_list: > + /* > + Only system variables can be used here. If this condition is changed > + please check careful code under lex->option_type == OPT_STATEMENT > + condition on wrong type casts. > + */ > + option_value_following_option_type > + | set_stmt_option_value_following_option_type_list ',' option_value_following_option_type > > // Start of option value list > start_option_value_list: > @@ -14522,6 +14570,14 @@ option_value_following_option_type: > { > LEX *lex= Lex; > > + /* > + Ignore SET STATEMENT variables list on slaves because system > + variables are not replicated except certain variables set the > + values of whose are written to binlog event header and nothing > + additional is required to set them. > + */ > + if (!(thd->slave_thread && lex->set_statement)) > + { Hmm, okay. Alternatively, you can simply make sure that set statement is not written into binlog. Or don't execute assignments in the slave thread. > if ($1.var && $1.var != trg_new_row_fake_var) > { > /* It is a system variable. */ Regards, Sergei

2 3

Re: [Maria-developers] [Commits] Rev 4445: MDEV-6878: Use of uninitialized saved_primary_key in Mrr_ordered_index_reader::resume_read() in file:///home/psergey/dev2/10.0/
by Jeremy Cole 20 Oct '14

20 Oct '14

Sergey, Nice test case. Would you mind cleaning up the end-space in your patch? Also, is there a reason to put the entire body of Mrr_ordered_index_reader::resume_read in the if() instead of an early return? Regards, Jeremy On Thu, Oct 16, 2014 at 11:58 AM, Sergey Petrunya <psergey(a)askmonty.org> wrote: > At file:///home/psergey/dev2/10.0/ > > ------------------------------------------------------------ > revno: 4445 > revision-id: psergey(a)askmonty.org-20141016135713-md92qr4g9fzhitxh > parent: bar(a)mnogosearch.org-20141013083155-j3j31hqhfc8i95y1 > committer: Sergey Petrunya <psergey(a)askmonty.org> > branch nick: 10.0 > timestamp: Thu 2014-10-16 17:57:13 +0400 > message: > MDEV-6878: Use of uninitialized saved_primary_key in > Mrr_ordered_index_reader::resume_read() > > - Make Mrr_ordered_index_reader::resume_read() restore index position > only if it was saved before with > Mrr_ordered_index_reader::interrupt_read(). > === modified file 'mysql-test/r/innodb_mrr_cpk.result' > --- a/mysql-test/r/innodb_mrr_cpk.result 2013-09-26 18:20:15 +0000 > +++ b/mysql-test/r/innodb_mrr_cpk.result 2014-10-16 13:57:13 +0000 > @@ -194,3 +194,41 @@ id select_type table type possible_keys > 2 DERIVED t2 ALL NULL NULL NULL NULL # > set join_cache_level= @tmp_mdev5037; > drop table t0,t1,t2; > +# > +# MDEV-6878: Use of uninitialized saved_primary_key in > Mrr_ordered_index_reader::resume_read() > +# > +create table t0(a int); > +insert into t0 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); > +create table t1 ( > +pk varchar(32) character set utf8 primary key, > +kp1 char(32) not null, > +col1 varchar(32), > +key (kp1) > +) engine=innodb; > +insert into t1 > +select > +concat('pk-', 1000 +A.a), > +concat('kp1-', 1000 +A.a), > +concat('val-', 1000 +A.a) > +from test.t0 A ; > +create table t2 as select kp1 as a from t1; > +set join_cache_level=8; > +set optimizer_switch='mrr=on,mrr_sort_keys=on'; > +explain > +select * from t2 straight_join t1 force index(kp1) where t1.kp1=t2.a; > +id select_type table type possible_keys key key_len > ref rows Extra > +1 SIMPLE t2 ALL NULL NULL NULL NULL 10 > +1 SIMPLE t1 ref kp1 kp1 32 test.t2.a 1 > Using join buffer (flat, BKAH join); Key-ordered Rowid-ordered scan > +select * from t2 straight_join t1 force index(kp1) where t1.kp1=t2.a; > +a pk kp1 col1 > +kp1-1000 pk-1000 kp1-1000 val-1000 > +kp1-1001 pk-1001 kp1-1001 val-1001 > +kp1-1002 pk-1002 kp1-1002 val-1002 > +kp1-1003 pk-1003 kp1-1003 val-1003 > +kp1-1004 pk-1004 kp1-1004 val-1004 > +kp1-1005 pk-1005 kp1-1005 val-1005 > +kp1-1006 pk-1006 kp1-1006 val-1006 > +kp1-1007 pk-1007 kp1-1007 val-1007 > +kp1-1008 pk-1008 kp1-1008 val-1008 > +kp1-1009 pk-1009 kp1-1009 val-1009 > +drop table t0,t1,t2; > > === modified file 'mysql-test/t/innodb_mrr_cpk.test' > --- a/mysql-test/t/innodb_mrr_cpk.test 2013-09-26 18:20:15 +0000 > +++ b/mysql-test/t/innodb_mrr_cpk.test 2014-10-16 13:57:13 +0000 > @@ -192,3 +192,35 @@ explain SELECT 1 FROM (SELECT url, id FR > set join_cache_level= @tmp_mdev5037; > > drop table t0,t1,t2; > + > +--echo # > +--echo # MDEV-6878: Use of uninitialized saved_primary_key in > Mrr_ordered_index_reader::resume_read() > +--echo # > +create table t0(a int); > +insert into t0 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); > + > +create table t1 ( > + pk varchar(32) character set utf8 primary key, > + kp1 char(32) not null, > + col1 varchar(32), > + key (kp1) > +) engine=innodb; > + > +insert into t1 > +select > + concat('pk-', 1000 +A.a), > + concat('kp1-', 1000 +A.a), > + concat('val-', 1000 +A.a) > +from test.t0 A ; > + > +create table t2 as select kp1 as a from t1; > + > +set join_cache_level=8; > +set optimizer_switch='mrr=on,mrr_sort_keys=on'; > +explain > +select * from t2 straight_join t1 force index(kp1) where t1.kp1=t2.a; > +select * from t2 straight_join t1 force index(kp1) where t1.kp1=t2.a; > + > +drop table t0,t1,t2; > + > + > > === modified file 'sql/multi_range_read.cc' > --- a/sql/multi_range_read.cc 2014-06-06 17:28:42 +0000 > +++ b/sql/multi_range_read.cc 2014-10-16 13:57:13 +0000 > @@ -426,6 +426,7 @@ bool Mrr_ordered_index_reader::set_inter > *space_start += key_len; > > have_saved_rowid= FALSE; > + read_was_interrupted= FALSE; > return FALSE; > } > > @@ -434,6 +435,7 @@ void Mrr_ordered_index_reader::set_no_in > support_scan_interruptions= FALSE; > saved_key_tuple= saved_rowid= saved_primary_key= NULL; /* safety */ > have_saved_rowid= FALSE; > + read_was_interrupted= FALSE; > } > > void Mrr_ordered_index_reader::interrupt_read() > @@ -451,6 +453,7 @@ void Mrr_ordered_index_reader::interrupt > &table->key_info[table->s->primary_key], > table->key_info[table->s->primary_key].key_length); > } > + read_was_interrupted= TRUE; > > /* Save the last rowid */ > memcpy(saved_rowid, file->ref, file->ref_length); > @@ -468,14 +471,17 @@ void Mrr_ordered_index_reader::position( > void Mrr_ordered_index_reader::resume_read() > { > TABLE *table= file->get_table(); > - KEY *used_index= &table->key_info[file->active_index]; > - key_restore(table->record[0], saved_key_tuple, > - used_index, used_index->key_length); > - if (saved_primary_key) > + if (read_was_interrupted) > { > - key_restore(table->record[0], saved_primary_key, > - &table->key_info[table->s->primary_key], > - table->key_info[table->s->primary_key].key_length); > + KEY *used_index= &table->key_info[file->active_index]; > + key_restore(table->record[0], saved_key_tuple, > + used_index, used_index->key_length); > + if (saved_primary_key) > + { > + key_restore(table->record[0], saved_primary_key, > + &table->key_info[table->s->primary_key], > + table->key_info[table->s->primary_key].key_length); > + } > } > } > > @@ -557,8 +563,7 @@ int Mrr_ordered_index_reader::init(handl > is_mrr_assoc= !MY_TEST(mode & HA_MRR_NO_ASSOCIATION); > mrr_funcs= *seq_funcs; > source_exhausted= FALSE; > - if (support_scan_interruptions) > - bzero(saved_key_tuple, key_info->key_length); > + read_was_interrupted= false; > have_saved_rowid= FALSE; > return 0; > } > > === modified file 'sql/multi_range_read.h' > --- a/sql/multi_range_read.h 2013-09-26 18:20:15 +0000 > +++ b/sql/multi_range_read.h 2014-10-16 13:57:13 +0000 > @@ -339,6 +339,12 @@ class Mrr_ordered_index_reader : public > > uchar *saved_key_tuple; /* Saved current key tuple */ > uchar *saved_primary_key; /* Saved current primary key tuple */ > + > + /* > + TRUE<=> saved_key_tuple (and saved_primary_key when applicable) have > + valid values. > + */ > + bool read_was_interrupted; > > static int compare_keys(void* arg, uchar* key1, uchar* key2); > static int compare_keys_reverse(void* arg, uchar* key1, uchar* key2); > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits >

2 1

Re: [Maria-developers] MariaDB 10.0.14 got rid of replication safety warning without warning in release notes?
by Pavel Ivanov 20 Oct '14

20 Oct '14

> Pavel> Which key is "the first duplicate key". > > The first key that has a duplicate value in the order of keys in the KEY > structure that MariaDB provides for the storage engine. I'm definitely not that familiar with the relevant MariaDB code, but I didn't find KEY structure to be passed to any insert-related function. Can you elaborate (preferably with code references) how this restriction is supposed to work? > This is the same order that keys are presented in SHOW CREATE TABLE Why is it safe to assume that keys have the same order on master and on slave? > To help engine writes we are constantly adding new test in the storage > engine test suite, to make it easier for storage engine vendors to > test that their storage engine follows the interface. > We have now added a test to check for the correct behaviour for the > case of INSERT ... ON DUPLICATE KEY. This is very interesting. Can you say which test it is and what exactly does it test? Is it present in 10.0.14 or it's only at the latest 10.0 branch? > Do you have or use a storage engine that doesn't provide the expected value > for the conflicting key ? I don't have a specific example of storage engine that detects duplicate key in different order, but I'm not convinced and don't have hard proof that such engine cannot exist and that InnoDB itself is not susceptible to this problem. I can easily imagine though that even given the particular order of keys in table definition the key reported as having duplicate can depend on the way storage engine stores those keys. BTW, can you point to MySQL documentation and/or comment in the code that indicates this requirement to storage engine implementation? > Pavel> On the related note Release Notes mention the error log flood > Pavel> protection. Why it's not behind flag? How should I disable the feature > Pavel> and get back to the old behavior? > > We already had in MariaDB flood protection for some error > messages. Can you say exactly which error messages are suppressed already? Is there a documentation link about that? > I am happy to look into adding a flag that is usable for any kind of > suppressed messages. > > A couple of ways to do this: > > a) Not suppress anything if you give --log-warnings a value > 2 > b) make LIMIT_UNSAFE_WARNING_ACTIVATION_TIMEOUT a configurable > variable, where 0 would mean no suppression. > > Which of the above, or both, would you prefer? > Or do you have any other suggestion of how to do this? Reading documentation on --log-warnings I don't see how it can be related to the error suppression. So I think making LIMIT_UNSAFE_WARNING_ACTIVATION_TIMEOUT configurable would be much better, though it should apply to all possible error suppressions. Thank you, Pavel On Fri, Oct 17, 2014 at 1:43 PM, Michael Widenius <michael.widenius(a)gmail.com> wrote: > > Hi! > > First sorry for the delayed reply; I am at a conference in Oxford and > didn't have an reliable internet connection until now. > >>>>>> "Pavel" == Pavel Ivanov <pivanof(a)google.com> writes: > > Pavel> Hi, > Pavel> I see that in MariaDB 10.0.14 Michael disabled replication safety > Pavel> warning for the case of INSERT ON DUPLICATE KEY UPDATE with several > Pavel> unique keys with a comment "We should assume that the store engine > Pavel> will report the first duplicate key for this case." > Pavel> http://bazaar.launchpad.net/~maria-captains/maria/10.0/revision/4400.1.3 > > Pavel> What does that mean? > > That we don't anymore give a warning for things that can't happen in > the distributed MariaDB server. > > Pabel> Which key is "the first duplicate key". > > The first key that has a duplicate value in the order of keys in the KEY > structure that MariaDB provides for the storage engine. > > This is the same order that keys are presented in SHOW CREATE TABLE > > Pavel> and why > Pavel> MariaDB relies on an engine behavior which AFAIK is not guaranteed? > > This is guaranteed for all storage engines shipped with MariaDB and > should also be guaranteed for any storage engine that wants to follow > the MariaDB and even the original MySQL storage engine interface. > > The change is also in line with how I designed the part of the storage > engine interface that gives the SQL layer the information about which > key that was conflicted. As we use this for things like INSERT ... ON > DUPLICATE KEY, it's critical that all storage engine provides the same > information in case of conflicts. > > To help engine writes we are constantly adding new test in the storage > engine test suite, to make it easier for storage engine vendors to > test that their storage engine follows the interface. > We have now added a test to check for the correct behaviour for the > case of INSERT ... ON DUPLICATE KEY. > > What is important when it comes to the storage engine interface is > that in all possible cases, a query should work identically against 2 > different storage engines. > > The only 'allowed difference's that I can think of for the moment are: > > - Different storage engines may retrieve rows in different order. > - In case of errors, changes to non transactional tables will not roll > back. > - Automatically generated key values may be different between storage engines. > (but preferably they should not be that). However when things are > replicated the slave, it should get the same values as the master. > > In the case of INSERT ... ON DUPLICATE KEY, it was clear that someone > had broke the storage engine interface, for no good reason. Instead > of users having to depend on 'unsafe, different and undocumented > behavior' for which row as updated, it's better to fix this issue once > and for all, especially as fixing this is trivial. > > Do you have or use a storage engine that doesn't provide the expected value > for the conflicting key ? > > If yes, there is a couple of ways to fix this. I am happy to talk with > you to find a solution that will satisfy your needs and ours. > > I underatand that in many cases it's not critical which unique key the > storage engine reports as duplicate. However in some cases, it's > critical to get the first key. > > I could add a new extra call to inform the engine that we now require > the first duplicate key and not just any conflicting key. This would > be used for example for INSERT ... ON DUPLICATE KEY ... > > I would prefer to have the check for which key was duplicated in the > storage engine. However, if critically needed, and assuming there is more > than one storage engine that needs this, we could add a check on the > handler level that will do a lookup on all unique keys to find the > first duplicate one. > > What solution would you prefer? > > Pavel> And why this serious change in behavior wasn't mentioned in the > Pavel> Release Notes? > > I didn't see it as a a serious change as it's only about removing a > warning that has no value for any normal MariaDB user (who only use > storage engines supplied with MariadB) and which actually caused > serious problems for some MariaDB users. > > That said, I agree it was a mistake from my part to not notice that > this was missing in the release notes. I will try to do better in the > future and check that the release notes correctly reflects any changes > that I have done. > > I appologize for missing this in this case. > > Pavel> On the related note Release Notes mention the error log flood > Pavel> protection. Why it's not behind flag? How should I disable the feature > Pavel> and get back to the old behavior? > > We already had in MariaDB flood protection for some error > messages. What I did was to make it more general and extend it to also > include other error messages, like the one we are discussing. As > there wasn't a flag for the old suppression, I didn't see a need for > adding a flag for the new suppression. > > I am happy to look into adding a flag that is usable for any kind of > suppressed messages. > > A couple of ways to do this: > > a) Not suppress anything if you give --log-warnings a value > 2 > b) make LIMIT_UNSAFE_WARNING_ACTIVATION_TIMEOUT a configurable > variable, where 0 would mean no suppression. > > Which of the above, or both, would you prefer? > Or do you have any other suggestion of how to do this? > > For 10.1 we will probably add a mechanism where you can permanently > suppress any specific error message to be written to the error log. > (They would still be given to the client). > > Any suggestion or wishes regarding this? > > Regards, > Monty

1 0

[Maria-developers] Igor please review: [Commits] Rev 4446: MDEV-6879: Dereference of NULL primary_file->table in DsMrr_impl::get_disk_sweep_mrr_cost()
by Sergey Petrunia 20 Oct '14

20 Oct '14

Hello Igor, Could you please review the below: At file:///home/psergey/dev2/10.0-cp2/ ------------------------------------------------------------ revno: 4446 revision-id: psergey at askmonty.org-20141020175620-vodbkx9ugmdhh8c1 parent: psergey at askmonty.org-20141016135713-md92qr4g9fzhitxh committer: Sergey Petrunya <psergey at askmonty.org> branch nick: 10.0-cp2 timestamp: Mon 2014-10-20 21:56:20 +0400 message: MDEV-6879: Dereference of NULL primary_file->table in DsMrr_impl::get_disk_sweep_mrr_cost() (Attempt #2) - Don't attempt to use BKA for materialized derived tables. The table is neither filled nor fully opened yet, so attempt to call handler->multi_range_read_info() causes crash. === modified file 'mysql-test/r/derived_opt.result' --- a/mysql-test/r/derived_opt.result 2013-06-18 15:21:00 +0000 +++ b/mysql-test/r/derived_opt.result 2014-10-20 17:56:20 +0000 @@ -352,4 +352,33 @@ pk pk 72 72 80 80 drop table t1, t2, t3, t4; +# +# MDEV-6879: Dereference of NULL primary_file->table in DsMrr_impl::get_disk_sweep_mrr_cost() +# +create table t1(a int); +insert into t1 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); +create table t2 (a int, b int, c text); +insert into t2 +select +A.a + B.a* 10, +A.a + B.a* 10, +'blob-data' +from t1 A, t1 B; +set @tmp_jcl= @@join_cache_level; +set @tmp_os= @@optimizer_switch; +set join_cache_level=6; +set @@optimizer_switch='derived_merge=on,derived_with_keys=on,mrr=on'; +explain +select * from +t1 join +(select * from t2 order by a limit 1000) as D1 +where +D1.a= t1.a; +id select_type table type possible_keys key key_len ref rows Extra +1 PRIMARY t1 ALL NULL NULL NULL NULL 10 Using where +1 PRIMARY <derived2> ref key0 key0 5 test.t1.a 10 +2 DERIVED t2 ALL NULL NULL NULL NULL 100 Using filesort +set join_cache_level=@tmp_jcl; +set optimizer_switch=@tmp_os; +drop table t1, t2; set optimizer_switch=@exit_optimizer_switch; === modified file 'mysql-test/t/derived_opt.test' --- a/mysql-test/t/derived_opt.test 2012-11-19 16:17:46 +0000 +++ b/mysql-test/t/derived_opt.test 2014-10-20 17:56:20 +0000 @@ -272,5 +272,34 @@ limit 10; drop table t1, t2, t3, t4; +--echo # +--echo # MDEV-6879: Dereference of NULL primary_file->table in DsMrr_impl::get_disk_sweep_mrr_cost() +--echo # +create table t1(a int); +insert into t1 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9); + +create table t2 (a int, b int, c text); +insert into t2 +select + A.a + B.a* 10, + A.a + B.a* 10, + 'blob-data' +from t1 A, t1 B; + +set @tmp_jcl= @@join_cache_level; +set @tmp_os= @@optimizer_switch; +set join_cache_level=6; +set @@optimizer_switch='derived_merge=on,derived_with_keys=on,mrr=on'; +explain +select * from + t1 join + (select * from t2 order by a limit 1000) as D1 +where + D1.a= t1.a; + +set join_cache_level=@tmp_jcl; +set optimizer_switch=@tmp_os; +drop table t1, t2; + # The following command must be the last one the file set optimizer_switch=@exit_optimizer_switch; === modified file 'sql/sql_select.cc' --- a/sql/sql_select.cc 2014-10-09 08:30:11 +0000 +++ b/sql/sql_select.cc 2014-10-20 17:56:20 +0000 @@ -10550,6 +10550,19 @@ uint check_join_cache_usage(JOIN_TAB *ta } /* + Don't use BKA for materialized tables. We could actually have a + meaningful use of BKA when linked join buffers are used. + + The problem is, the temp.table is not filled (actually not even opened + properly) yet, and this doesn't let us call + handler->multi_range_read_info(). It is possible to come up with + estimates, etc. without acessing the table, but it seems not to worth the + effort now. + */ + if (tab->table->pos_in_table_list->is_materialized_derived()) + no_bka_cache= true; + + /* Don't use join buffering if we're dictated not to by no_jbuf_after (This is not meaningfully used currently) */ @@ -10615,7 +10628,7 @@ uint check_join_cache_usage(JOIN_TAB *ta if (tab->ref.is_access_triggered()) goto no_join_cache; - if (!tab->is_ref_for_hash_join()) + if (!tab->is_ref_for_hash_join() && !no_bka_cache) { flags= HA_MRR_NO_NULL_ENDPOINTS | HA_MRR_SINGLE_POINT; if (tab->table->covering_keys.is_set(tab->ref.key)) BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] review MDEV 6631
by Roberto Spadim 18 Oct '14

18 Oct '14

hi guys could review MDEV-6631? there's a pull request, maybe should include some test files, but I don't know how to implement profiler check with mysql test tools -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

Re: [Maria-developers] MariaDB 10.0.14 got rid of replication safety warning without warning in release notes?
by Pavel Ivanov 18 Oct '14

18 Oct '14

As I have reports of people not seeing this email on maria-developers@ list (probably because Monty sent it from the wrong address) I'm forwarding it to the list. I'll respond to it in detail later. On Fri, Oct 17, 2014 at 1:43 PM, Michael Widenius <michael.widenius(a)gmail.com> wrote: > > Hi! > > First sorry for the delayed reply; I am at a conference in Oxford and > didn't have an reliable internet connection until now. > >>>>>> "Pavel" == Pavel Ivanov <pivanof(a)google.com> writes: > > Pavel> Hi, > Pavel> I see that in MariaDB 10.0.14 Michael disabled replication safety > Pavel> warning for the case of INSERT ON DUPLICATE KEY UPDATE with several > Pavel> unique keys with a comment "We should assume that the store engine > Pavel> will report the first duplicate key for this case." > Pavel> http://bazaar.launchpad.net/~maria-captains/maria/10.0/revision/4400.1.3 > > Pavel> What does that mean? > > That we don't anymore give a warning for things that can't happen in > the distributed MariaDB server. > > Pabel> Which key is "the first duplicate key". > > The first key that has a duplicate value in the order of keys in the KEY > structure that MariaDB provides for the storage engine. > > This is the same order that keys are presented in SHOW CREATE TABLE > > Pavel> and why > Pavel> MariaDB relies on an engine behavior which AFAIK is not guaranteed? > > This is guaranteed for all storage engines shipped with MariaDB and > should also be guaranteed for any storage engine that wants to follow > the MariaDB and even the original MySQL storage engine interface. > > The change is also in line with how I designed the part of the storage > engine interface that gives the SQL layer the information about which > key that was conflicted. As we use this for things like INSERT ... ON > DUPLICATE KEY, it's critical that all storage engine provides the same > information in case of conflicts. > > To help engine writes we are constantly adding new test in the storage > engine test suite, to make it easier for storage engine vendors to > test that their storage engine follows the interface. > We have now added a test to check for the correct behaviour for the > case of INSERT ... ON DUPLICATE KEY. > > What is important when it comes to the storage engine interface is > that in all possible cases, a query should work identically against 2 > different storage engines. > > The only 'allowed difference's that I can think of for the moment are: > > - Different storage engines may retrieve rows in different order. > - In case of errors, changes to non transactional tables will not roll > back. > - Automatically generated key values may be different between storage engines. > (but preferably they should not be that). However when things are > replicated the slave, it should get the same values as the master. > > In the case of INSERT ... ON DUPLICATE KEY, it was clear that someone > had broke the storage engine interface, for no good reason. Instead > of users having to depend on 'unsafe, different and undocumented > behavior' for which row as updated, it's better to fix this issue once > and for all, especially as fixing this is trivial. > > Do you have or use a storage engine that doesn't provide the expected value > for the conflicting key ? > > If yes, there is a couple of ways to fix this. I am happy to talk with > you to find a solution that will satisfy your needs and ours. > > I underatand that in many cases it's not critical which unique key the > storage engine reports as duplicate. However in some cases, it's > critical to get the first key. > > I could add a new extra call to inform the engine that we now require > the first duplicate key and not just any conflicting key. This would > be used for example for INSERT ... ON DUPLICATE KEY ... > > I would prefer to have the check for which key was duplicated in the > storage engine. However, if critically needed, and assuming there is more > than one storage engine that needs this, we could add a check on the > handler level that will do a lookup on all unique keys to find the > first duplicate one. > > What solution would you prefer? > > Pavel> And why this serious change in behavior wasn't mentioned in the > Pavel> Release Notes? > > I didn't see it as a a serious change as it's only about removing a > warning that has no value for any normal MariaDB user (who only use > storage engines supplied with MariadB) and which actually caused > serious problems for some MariaDB users. > > That said, I agree it was a mistake from my part to not notice that > this was missing in the release notes. I will try to do better in the > future and check that the release notes correctly reflects any changes > that I have done. > > I appologize for missing this in this case. > > Pavel> On the related note Release Notes mention the error log flood > Pavel> protection. Why it's not behind flag? How should I disable the feature > Pavel> and get back to the old behavior? > > We already had in MariaDB flood protection for some error > messages. What I did was to make it more general and extend it to also > include other error messages, like the one we are discussing. As > there wasn't a flag for the old suppression, I didn't see a need for > adding a flag for the new suppression. > > I am happy to look into adding a flag that is usable for any kind of > suppressed messages. > > A couple of ways to do this: > > a) Not suppress anything if you give --log-warnings a value > 2 > b) make LIMIT_UNSAFE_WARNING_ACTIVATION_TIMEOUT a configurable > variable, where 0 would mean no suppression. > > Which of the above, or both, would you prefer? > Or do you have any other suggestion of how to do this? > > For 10.1 we will probably add a mechanism where you can permanently > suppress any specific error message to be written to the error log. > (They would still be given to the client). > > Any suggestion or wishes regarding this? > > Regards, > Monty

1 0

Re: [Maria-developers] [Commits] Rev 3900: Fix for test failures on 64-bit platform. in lp:~maria-captains/maria/10.0-galera-6593
by Kristian Nielsen 18 Oct '14

18 Oct '14

Nirbhay Choubey <nirbhay(a)mariadb.com> writes: > message: > Fix for test failures on 64-bit platform. > === modified file 'sql/rpl_mi.cc' > --- a/sql/rpl_mi.cc 2014-10-10 23:06:40 +0000 > +++ b/sql/rpl_mi.cc 2014-10-17 02:08:55 +0000 > @@ -1466,7 +1466,7 @@ > { > for (int i= DO_DOMAIN_IDS; i <= IGNORE_DOMAIN_IDS; i ++) > { > - my_init_dynamic_array(&m_domain_ids[i], sizeof(uint32), 16, 16, MYF(0)); > + my_init_dynamic_array(&m_domain_ids[i], sizeof(ulong), 16, 16, MYF(0)); > } > } > Why do you need to use ulong? Domain ids should always be uint32, also on 64-bit. What is the real problem? Eg. why is there a test failure? I'd expect that the right fix was to convert to uint32 in the places where a domain_id might come into the code as ulong... > @@ -1483,7 +1483,7 @@ > domain ids list. DO_DOMAIN_IDS list is only looked-up is both (do & ignore) > list are non-empty. > */ > -void Domain_id_filter::do_filter(uint32 domain_id) > +void Domain_id_filter::do_filter(ulong domain_id) > { > DYNAMIC_ARRAY *do_domain_ids= &m_domain_ids[DO_DOMAIN_IDS]; > DYNAMIC_ARRAY *ignore_domain_ids= &m_domain_ids[IGNORE_DOMAIN_IDS]; > @@ -1491,21 +1491,21 @@ > if (do_domain_ids->elements > 0) > { > if (likely(do_domain_ids->elements == 1)) > - m_filter= ((* (uint32*) dynamic_array_ptr(do_domain_ids, 0)) > + m_filter= ((* (ulong *) dynamic_array_ptr(do_domain_ids, 0)) > != domain_id); > else > - m_filter= (bsearch((const uint32 *) &domain_id, do_domain_ids->buffer, > - do_domain_ids->elements, sizeof(uint32), > + m_filter= (bsearch((const ulong *) &domain_id, do_domain_ids->buffer, > + do_domain_ids->elements, sizeof(ulong), Why is a cast needed here? If it's because of const, could you avoid the cast just by declaring domain_id const in the argument of the function? Thanks, - Kristian.

2 1

[Maria-developers] doubts about show ___ any information table plugin
by Roberto Spadim 17 Oct '14

17 Oct '14

hi guys ! :) i was reading this: https://mariadb.com/kb/en/mariadb/development/mariadb-internals-documentati… and i'm with some doubts... automatically parser translate "_" to space? for example table A_B, i could use SHOW A B ? about the FLUSH what it does? any example of what it should do? like... i know we can reset some information counters, it's a 'shortcut' to avoid something like "SET pluginvariable=0"? if yes, maybe we could add some example at KB page, i'm with doubts, maybe a dba without knowledge about plugin internal will not understand thanks :) -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

2 1

[Maria-developers] help with auth plugins
by Roberto Spadim 17 Oct '14

17 Oct '14

Hi guys! :) i need some help... i got the auth_dialog and i'm including some features to implement otp (one time password) my question is... what ".h" file should i include to allow read/write from any_table from mysql schema? i checked some read/write from selftunning optimizer ( https://github.com/igniting/server/blob/3d101751d27215ccf88c6c9701ca113de4a… ) but i didn't understand what should be included i'm getting messages like: In file included from /home/rspadim/mariadb/mariadb-10.1/server/sql/opt_range.h:36:0, from /home/rspadim/mariadb/mariadb-10.1/server/plugin/auth_otp/otp_auth.cc:26: --- my current source code (i don't know if .h order is ok): INCLUDES: #include "unireg.h" <- line 20 #include "sql_base.h" #include "lex.h" #include "sql_table.h" #include "sql_priv.h" #include "sql_class.h" #include "opt_range.h" <- line 26 #include "records.h" #include <mysql/plugin_auth.h> #include <string.h> #include <stdio.h> #include <stdlib.h> #include <mysql/auth_dialog_client.h> -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

[Maria-developers] Two bugs in MRR in need of review/comment
by Jeremy Cole 16 Oct '14

16 Oct '14

Hello, I filed these two bugs against multi-range read (MRR) in MariaDB 10.0.12 which affected us and could use some review of the patch for correctness, along with development of a proper test case: MDEV-6878 <https://mariadb.atlassian.net/browse/MDEV-6878>: Use of uninitialized saved_primary_key in Mrr_ordered_index_reader::resume_read() MDEV-6879 <https://mariadb.atlassian.net/browse/MDEV-6879>: Dereference of NULL primary_file->table in DsMrr_impl::get_disk_sweep_mrr_cost() Could someone please take a look at them? Regards, Jeremy

1 0

Re: [Maria-developers] MDEV-5359 CREATE OR REPLACE, CREATE IF NOT EXISTS, DROP IF EXISTS
by Sergei Golubchik 15 Oct '14

15 Oct '14

Hi, Alexander! This is all pretty good. Great work, thanks. I didn't have many comments. Few about small stylistic changes, and a couple of more serious ones - about tests and about events. See below. On Oct 02, Alexander Barkov wrote: > diff --git a/mysql-test/r/create_drop_db.result b/mysql-test/r/create_drop_db.result > new file mode 100644 > index 0000000..e7d4c18 > --- /dev/null > +++ b/mysql-test/r/create_drop_db.result > @@ -0,0 +1,17 @@ > +CREATE DATABASE db1; > +CREATE DATABASE IF NOT EXISTS db1; > +Warnings: > +Note 1007 Can't create database 'db1'; database exists > +CREATE OR REPLACE DATABASE db1; > +CREATE OR REPLACE DATABASE IF NOT EXISTS db2; > +ERROR HY000: Incorrect usage of OR REPLACE and IF NOT EXISTS > +DROP DATABASE db1; > +DROP DATABASE IF EXISTS db1; > +Warnings: > +Note 1008 Can't drop database 'db1'; database doesn't exist > +DROP DATABASE db1; > +ERROR HY000: Can't drop database 'db1'; database doesn't exist > +CREATE OR REPLACE DATABASE db1; > +CREATE DATABASE db1; > +ERROR HY000: Can't create database 'db1'; database exists > +DROP DATABASE IF EXISTS db1; I don't see where this test verifies that CREATE OR REPLACE actually replaces a database. Add the following: create database; create table; create or replace database; the table must not exist now; and add a test for create if not exists when a database does not exist. you can combine it with a previous test, if you'd like. > diff --git a/mysql-test/r/create_drop_event.result b/mysql-test/r/create_drop_event.result > new file mode 100644 > index 0000000..40d16e6 > --- /dev/null > +++ b/mysql-test/r/create_drop_event.result > @@ -0,0 +1,31 @@ > +SET GLOBAL event_scheduler=off; > +CREATE DATABASE db1; > +CREATE EVENT event_1 ON SCHEDULE EVERY 1 SECOND DO DROP DATABASE db1; > +SHOW DATABASES LIKE 'db1'; > +Database (db1) > +db1 > +SET GLOBAL event_scheduler=on; > +SHOW DATABASES LIKE 'db1'; > +Database (db1) > +SET GLOBAL event_scheduler=off; > +DROP DATABASE db1; > +ERROR HY000: Can't drop database 'db1'; database doesn't exist > +CREATE EVENT event_1 ON SCHEDULE EVERY 1 SECOND DO DROP DATABASE db1; > +ERROR HY000: Event 'event_1' already exists > +CREATE EVENT IF NOT EXISTS event_1 ON SCHEDULE EVERY 1 SECOND DO DROP DATABASE db1; > +Warnings: > +Note 1537 Event 'event_1' already exists > +CREATE OR REPLACE EVENT IF NOT EXISTS event_1 ON SCHEDULE EVERY 1 SECOND DO DROP DATABASE db1; > +ERROR HY000: Incorrect usage of OR REPLACE and IF NOT EXISTS > +CREATE OR REPLACE EVENT event_1 ON SCHEDULE EVERY 1 SECOND DO DROP DATABASE db1; again, same thing. I don't see a test that the command actually worked. please use different event definitions in all three create variants. and add show create event after every create to verify that create if not exists did *not* modify the event, but create or replace did modify it. > +CREATE DATABASE db1; > +SHOW DATABASES LIKE 'db1'; > +Database (db1) > +db1 > +SET GLOBAL event_scheduler=on; > +SHOW DATABASES LIKE 'db1'; > +Database (db1) > +SET GLOBAL event_scheduler=off; > +DROP DATABASE db1; > +ERROR HY000: Can't drop database 'db1'; database doesn't exist > +DROP EVENT IF EXISTS event_1; > diff --git a/mysql-test/r/create_drop_function.result b/mysql-test/r/create_drop_function.result > new file mode 100644 > index 0000000..750d179 > --- /dev/null > +++ b/mysql-test/r/create_drop_function.result > @@ -0,0 +1,34 @@ > +SET timestamp=UNIX_TIMESTAMP('2014-09-30 08:00:00'); > +CREATE FUNCTION f1(str char(20)) > +RETURNS CHAR(100) > +RETURN CONCAT('Hello, ', str, '!'); > +SELECT * FROM mysql.proc WHERE name like 'f1'; > +db name type specific_name language sql_data_access is_deterministic security_type param_list returns body definer created modified sql_mode comment character_set_client collation_connection db_collation body_utf8 > +test f1 FUNCTION f1 SQL CONTAINS_SQL NO DEFINER str char(20) char(100) CHARSET latin1 RETURN CONCAT('Hello, ', str, '!') root@localhost 2014-09-30 08:00:00 2014-09-30 08:00:00 latin1 latin1_swedish_ci latin1_swedish_ci RETURN CONCAT('Hello, ', str, '!') > +SELECT f1('world'); > +f1('world') > +Hello, world! > +CREATE FUNCTION f1(str char(20)) > +RETURNS TEXT > +RETURN CONCAT('Hello, ', str, '!'); > +ERROR 42000: FUNCTION f1 already exists > +CREATE FUNCTION IF NOT EXISTS f1(str char(20)) > +RETURNS CHAR(100) > +RETURN CONCAT('Hello, ', str, '!'); > +Warnings: > +Note 1304 FUNCTION f1 already exists same. please *different* definitions in different create variants. and some verification that the definition changed (or didn't change) - like show create or select f1(); > +CREATE OR REPLACE FUNCTION IF NOT EXISTS f1(str char(20)) > +RETURNS CHAR(100) > +RETURN CONCAT('Hello, ', str, '!'); > +ERROR HY000: Incorrect usage of OR REPLACE and IF NOT EXISTS > +CREATE OR REPLACE FUNCTION f1(str char(20)) > +RETURNS CHAR(100) > +RETURN CONCAT('Hello, ', str, '!'); > +DROP FUNCTION f1; > +CREATE FUNCTION IF NOT EXISTS f1(str char(20)) > +RETURNS CHAR(100) > +RETURN CONCAT('Hello, ', str, '!'); > +SELECT f1('world'); > +f1('world') > +Hello, world! > +DROP FUNCTION f1; > diff --git a/mysql-test/r/create_drop_index.result b/mysql-test/r/create_drop_index.result > new file mode 100644 > index 0000000..23f6388 > --- /dev/null > +++ b/mysql-test/r/create_drop_index.result > @@ -0,0 +1,34 @@ > +CREATE TABLE t1(id INT); > +CREATE INDEX idx ON t1(id); > +ALTER TABLE t1 ADD INDEX idx(id); > +ERROR 42000: Duplicate key name 'idx' > +ALTER TABLE t1 ADD INDEX IF NOT EXISTS idx(id); > +Warnings: > +Note 1061 Duplicate key name 'idx' > +CREATE OR REPLACE INDEX idx2 ON t1(id); same comment here (and everywhere below) > +Warnings: > +Note 1831 Duplicate index 'idx2' defined on the table 'test.t1'. This is deprecated and will be disallowed in a future release. > +ALTER TABLE t1 ADD INDEX IF NOT EXISTS idx2(id); > +Warnings: > +Note 1061 Duplicate key name 'idx2' > +CREATE OR REPLACE UNIQUE INDEX idx ON t1(id); > +CREATE INDEX idx ON t1(id); > +ERROR 42000: Duplicate key name 'idx' > +CREATE TABLE t2(col INT); > +CREATE OR REPLACE INDEX idx ON t2(col); > +CREATE INDEX idx ON t2(col); > +ERROR 42000: Duplicate key name 'idx' > +# Is this correct behaviour? > +ALTER TABLE t2 ADD FOREIGN KEY fk(col) REFERENCES t1(id); > +CREATE INDEX fk ON t2(col); > +Warnings: > +Note 1831 Duplicate index 'fk' defined on the table 'test.t2'. This is deprecated and will be disallowed in a future release. > +# Clearing up > +DROP INDEX idx ON t1; > +DROP TABLE t1; > +ALTER TABLE t1 DROP INDEX idx2; > +ERROR 42S02: Table 'test.t1' doesn't exist > +ALTER TABLE t2 DROP INDEX idx; > +ALTER TABLE t2 DROP INDEX fk; > +ALTER TABLE t2 DROP FOREIGN KEY fk; > +DROP TABLE t2; > diff --git a/mysql-test/r/proc_func.result b/mysql-test/r/proc_func.result > new file mode 100644 > index 0000000..1105ded > --- /dev/null > +++ b/mysql-test/r/proc_func.result > @@ -0,0 +1,63 @@ > +# > +# Testing CREATE PROCEDURE IF NOT EXISTS > +# > +DROP TABLE IF EXISTS t1; > +DROP PROCEDURE IF EXISTS proc1; > +CREATE TABLE t1 (id INT); > +CREATE PROCEDURE proc1 (OUT cnt INT) BEGIN SELECT COUNT(*) INTO cnt FROM t1; END$$ how is this test (and other tests below) different from create_drop_xxx tests above? > +CALL proc1(@cnt); > +SELECT @cnt; > +@cnt > +0 > +INSERT INTO t1 VALUES (1), (2), (3); > +CALL proc1(@cnt); > +SELECT @cnt; > +@cnt > +3 > +CREATE PROCEDURE proc1 (OUT cnt INT) BEGIN SELECT COUNT(*) INTO cnt FROM t1; END$$ > diff --git a/sql/event_db_repository.cc b/sql/event_db_repository.cc > index b20269b..01cdabb 100644 > --- a/sql/event_db_repository.cc > +++ b/sql/event_db_repository.cc > @@ -684,18 +684,28 @@ Event_db_repository::create_event(THD *thd, Event_parse_data *parse_data, > DBUG_PRINT("info", ("check existance of an event with the same name")); > if (!find_named_event(parse_data->dbname, parse_data->name, table)) > { > - if (create_if_not) > + if (thd->lex->is_create_or_replace()) > + { > + if ((ret= table->file->ha_delete_row(table->record[0]))) > + { > + table->file->print_error(ret, MYF(0)); > + goto end; > + } > + } > + else if (create_if_not) > { > *event_already_exists= true; > push_warning_printf(thd, Sql_condition::WARN_LEVEL_NOTE, > ER_EVENT_ALREADY_EXISTS, ER(ER_EVENT_ALREADY_EXISTS), > parse_data->name.str); > ret= 0; > + goto end; > } > else > + { > my_error(ER_EVENT_ALREADY_EXISTS, MYF(0), parse_data->name.str); > - > - goto end; > + goto end; > + } > } else > *event_already_exists= false; I wonder whether this works as it should. On create-or-replace you delete the old event row and a new one is supposedly inserted. So, the table is fine. But event_already_exists will stay unset, undefined, actually, because the caller didn't set it either. If you set it to TRUE, then the caller won't update in-memory even list, so your new event won't be enabled. If you set it to FALSE, it will try to insert a new event without deleting the old one and I don't know what will happen (either both will stay active or the insertion fails). Do you have a test to verify that after CREATE OR REPLACE EVENT the old event no longer exists and the new event does exist and is executed as expected? > diff --git a/sql/share/errmsg-utf8.txt b/sql/share/errmsg-utf8.txt > index ed7976e..1cfaaa4 100644 > --- a/sql/share/errmsg-utf8.txt > +++ b/sql/share/errmsg-utf8.txt > @@ -7110,3 +7110,11 @@ ER_IT_IS_A_VIEW 42S02 > eng "'%-.192s' is a view" > ER_SLAVE_SKIP_NOT_IN_GTID > eng "When using GTID, @@sql_slave_skip_counter can not be used. Instead, setting @@gtid_slave_pos explicitly can be used to skip to after a given GTID position." > +ER_USER_CREATE_EXISTS > + eng "Can't create user '%-.64s'@'%-.64s'; it already exists" > +ER_USER_DROP_EXISTS > + eng "Can't drop user '%-.64s'@'%-.64s'; it doesn't exist" > +ER_ROLE_CREATE_EXISTS > + eng "Can't create role '%-.64s'; it already exists" > +ER_ROLE_DROP_EXISTS > + eng "Can't drop role '%-.64s'; it doesn't exist" just a reminder - when merging (preferrably - rebasing) don't forget to put these errror messages after all 10.0 error messages. > diff --git a/sql/sp.cc b/sql/sp.cc > index 188b311..058a617 100644 > --- a/sql/sp.cc > +++ b/sql/sp.cc > @@ -835,6 +835,8 @@ db_load_routine(THD *thd, stored_procedure_type type, > > thd->lex= &newlex; > newlex.current_select= NULL; > + // Resetting all the flags in create_info > + newlex.create_info.options= 0; Isn't it safer to reset options in the parser, after CREATE clause? Like | CREATE { Lex->create_info.options= 0; } opt_or_replace ... Ok, see my very last comment in this review. > defstr.set_charset(creation_ctx->get_client_cs()); > > @@ -1015,6 +1063,47 @@ sp_create_routine(THD *thd, stored_procedure_type type, sp_head *sp) > ret= SP_OPEN_TABLE_FAILED; > else > { > + /* Checking if the routine already exists */ > + if (db_find_routine_aux(thd, type, lex->spname, table) == SP_OK) > + { > + if (lex->is_create_or_replace()) > + { > + if ((ret= sp_drop_routine_internal(thd, type, lex->spname, table))) > + goto done; > + } > + else if (lex->is_create_if_not_exists()) > + { > + push_warning_printf(thd, Sql_condition::WARN_LEVEL_NOTE, > + ER_SP_ALREADY_EXISTS, ER(ER_SP_ALREADY_EXISTS), > + type == TYPE_ENUM_FUNCTION ? > + "FUNCTION" : "PROCEDURE", > + lex->spname->m_name.str); > + > + ret= SP_OK; > + > + // Setting retstr as it is used for logging. > + if (sp->m_type == TYPE_ENUM_FUNCTION) > + { > + sp_returns_type(thd, retstr, sp); this is ok > + store_failed= store_failed || > + table->field[MYSQL_PROC_FIELD_RETURNS]-> > + store(retstr.ptr(), retstr.length(), system_charset_info); but why do you need to update table->field[MYSQL_PROC_FIELD_RETURNS] ? > + } > + > + if (store_failed) > + { > + ret= SP_FLD_STORE_FAILED; > + goto done; > + } > + goto log; > + } > + else > + { > + ret= SP_WRITE_ROW_FAILED; > + goto done; > + } > + } > + > restore_record(table, s->default_values); // Get default values for fields > > /* NOTE: all needed privilege checks have been already done. */ > diff --git a/sql/sql_acl.cc b/sql/sql_acl.cc > index 5f077c2..1e156be 100644 > --- a/sql/sql_acl.cc > +++ b/sql/sql_acl.cc > @@ -9148,6 +9148,42 @@ end: > DBUG_RETURN(result); > } > > + > +class Warning_user > +{ > +private: > + bool m_handle_as_role; > + void push_warning_role(THD *thd, uint err, const LEX_USER *user) > + { > + push_warning_printf(thd, Sql_condition::WARN_LEVEL_NOTE, > + err, ER(err), user->user.str); > + } > + void push_warning_user(THD *thd, uint err, const LEX_USER *user) > + { > + push_warning_printf(thd, Sql_condition::WARN_LEVEL_NOTE, > + err, ER(err), user->user.str, user->host.str); > + } > +public: > + Warning_user(bool handle_as_role) > + :m_handle_as_role(handle_as_role) > + { } > + void push_create_exists(THD *thd, const LEX_USER *user) > + { > + if (m_handle_as_role) > + push_warning_role(thd, ER_ROLE_CREATE_EXISTS, user); > + else > + push_warning_user(thd, ER_USER_CREATE_EXISTS, user); > + } > + void push_drop_doesnt_exist(THD *thd, const LEX_USER *user) > + { > + if (m_handle_as_role) > + push_warning_role(thd, ER_ROLE_DROP_EXISTS, user); > + else > + push_warning_user(thd, ER_USER_DROP_EXISTS, user); > + } > +}; > + > + > /* > Create a list of users. > > @@ -9215,10 +9252,26 @@ bool mysql_create_user(THD *thd, List <LEX_USER> &list, bool handle_as_role) > */ > if (handle_grant_data(tables, 0, user_name, NULL)) > { > - append_user(thd, &wrong_users, user_name); > - > - result= TRUE; > - continue; > + if (thd->lex->is_create_or_replace()) > + { > + if (handle_grant_data(tables, 1, user_name, NULL) <= 0) > + { > + append_user(thd, &wrong_users, user_name); I don't understand that at all. If user creation failed, you try to drop it, but don't create a user again, after the old one was dropped. How could it work? > + result= TRUE; > + continue; > + } > + } > + else if (thd->lex->is_create_if_not_exists()) > + { > + Warning_user(handle_as_role).push_create_exists(thd, user_name); Eh? Why do you need a class for that? Especially as it's temporarily instantiated here, and just to push a warning. Please, remove it. > + continue; > + } > + else > + { > + append_user(thd, &wrong_users, user_name); > + result= TRUE; > + continue; > + } > } > > some_users_created= TRUE; > @@ -9268,7 +9321,7 @@ bool mysql_create_user(THD *thd, List <LEX_USER> &list, bool handle_as_role) > (handle_as_role) ? "CREATE ROLE" : "CREATE USER", > wrong_users.c_ptr_safe()); > > - if (some_users_created) > + if (some_users_created || !result) What is that for? To binlog CREATE IF NOT EXISTS? > result |= write_bin_log(thd, FALSE, thd->query(), thd->query_length()); > > mysql_rwlock_unlock(&LOCK_grant); > diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc > index 922f0d7..648fd48 100644 > --- a/sql/sql_parse.cc > +++ b/sql/sql_parse.cc > @@ -3855,7 +3855,9 @@ end_with_restore_list: > } > } > #endif > - if (check_access(thd, CREATE_ACL, lex->name.str, NULL, NULL, 1, 0)) > + if (check_access(thd, lex->is_create_or_replace() ? > + CREATE_ACL | DROP_ACL : CREATE_ACL, good > + lex->name.str, NULL, NULL, 1, 0)) > break; > res= mysql_create_db(thd, lex->name.str, &create_info, 0); > break; > @@ -3995,9 +3997,8 @@ end_with_restore_list: > switch (lex->sql_command) { > case SQLCOM_CREATE_EVENT: > { > - bool if_not_exists= (lex->create_info.options & > - HA_LEX_CREATE_IF_NOT_EXISTS); > - res= Events::create_event(thd, lex->event_parse_data, if_not_exists); > + res= Events::create_event(thd, lex->event_parse_data, > + lex->is_create_if_not_exists()); This doesn't seem to be particularly logical. You pass lex->is_create_if_not_exists)() here an argument, but you check thd->lex->is_create_or_replace() in the Event_db_repository::create_event anyway. I'd suggest not to pass lex->is_create_if_not_exists() as an argument, and check both in Event_db_repository::create_event(). > break; > } > case SQLCOM_ALTER_EVENT: > @@ -6514,7 +6520,11 @@ bool add_field_to_list(THD *thd, LEX_STRING *field_name, enum_field_types type, > lex->col_list.push_back(new Key_part_spec(*field_name, 0)); > key= new Key(Key::PRIMARY, null_lex_str, > &default_key_create_info, > - 0, lex->col_list, NULL, lex->check_exists); > + 0, lex->col_list, NULL); > + > + if (lex->check_exists) > + key->key_create_info.options = HA_LEX_CREATE_IF_NOT_EXISTS; Hmm, is it for ALTER TABLE t1 ADD COLUMN IF NOT EXISTS foo INT PRIMARY KEY ? Do you have test cases for it? What does it do if the column exists but the key doesn't? What if the key exists and the column doens't? > + > lex->alter_info.key_list.push_back(key); > lex->col_list.empty(); > } > @@ -6524,7 +6534,11 @@ bool add_field_to_list(THD *thd, LEX_STRING *field_name, enum_field_types type, > lex->col_list.push_back(new Key_part_spec(*field_name, 0)); > key= new Key(Key::UNIQUE, null_lex_str, > &default_key_create_info, 0, > - lex->col_list, NULL, lex->check_exists); > + lex->col_list, NULL); > + > + if (lex->check_exists) > + key->key_create_info.options = HA_LEX_CREATE_IF_NOT_EXISTS; Same, needs tests (if there aren't). > + > lex->alter_info.key_list.push_back(key); > lex->col_list.empty(); > } > diff --git a/sql/sql_table.cc b/sql/sql_table.cc > index 8af77d9..b10c6d7 100644 > --- a/sql/sql_table.cc > +++ b/sql/sql_table.cc > @@ -5846,6 +5846,8 @@ handle_if_exists_options(THD *thd, TABLE *table, Alter_info *alter_info) > } > if (remove_key) > { take care when merging. "if (remove_key)" is not in the current 10.1, instead there's "remove_key:" label and goto. > + if (key->is_create_if_not_exists()) > + { > push_warning_printf(thd, Sql_condition::WARN_LEVEL_NOTE, > ER_DUP_KEYNAME, ER(ER_DUP_KEYNAME), keyname); > key_it.remove(); > diff --git a/sql/sql_trigger.cc b/sql/sql_trigger.cc > index 70ac626..b337451 100644 > --- a/sql/sql_trigger.cc > +++ b/sql/sql_trigger.cc > @@ -607,6 +607,89 @@ end: > DBUG_RETURN(result); > } > > +/** > + Build stmt_query to write it in the bin-log. > + > + @param thd current thread context (including trigger definition in > + LEX) > + @param tables table list containing one open table for which the > + trigger is created. > + @param[out] stmt_query after successful return, this string contains > + well-formed statement for creation this trigger. > + > + @note > + - Assumes that trigger name is fully qualified. > + - NULL-string means the following LEX_STRING instance: > + { str = 0; length = 0 }. > + - In other words, definer_user and definer_host should contain > + simultaneously NULL-strings (non-SUID/old trigger) or valid strings > + (SUID/new trigger). > + > + @retval > + False success > + @retval > + True error > +*/ > +static bool build_trig_stmt_query(THD *thd, TABLE_LIST *tables, > + String *stmt_query, > + LEX_STRING *trg_definer, > + char trg_definer_holder[]) > +{ > + LEX *lex = thd->lex; > + LEX_STRING definer_user; > + LEX_STRING definer_host; > + > + if (trg_definer == NULL && > + !(trg_definer= alloc_lex_string(&tables->table->mem_root))) > + return true; > + > + if (lex->sphead->m_chistics->suid != SP_IS_NOT_SUID) > + { > + /* SUID trigger. */ > + definer_user= lex->definer->user; > + definer_host= lex->definer->host; > + lex->definer->set_lex_string(trg_definer, trg_definer_holder); > + } > + else > + { > + /* non-SUID trigger. */ > + definer_user.str= 0; > + definer_user.length= 0; > + > + definer_host.str= 0; > + definer_host.length= 0; > + > + trg_definer->str= (char*) ""; > + trg_definer->length= 0; Why do you bother setting definer here? You can simply do below (see below): > + } > + > + /* > + Create well-formed trigger definition query. Original query is not > + appropriated, because definer-clause can be not truncated. > + */ > + > + stmt_query->append(STRING_WITH_LEN("CREATE ")); > + > + if (lex->is_create_or_replace()) > + stmt_query->append(STRING_WITH_LEN("OR REPLACE ")); > + > + if (lex->sphead->m_chistics->suid != SP_IS_NOT_SUID) > + { > + /* > + Append definer-clause if the trigger is SUID (a usual trigger in > + new MySQL versions). > + */ > + append_definer(thd, stmt_query, &definer_user, &definer_host); here: append_definer(thd, stmt_query, &lex->definer->user, &lex->definer->host); and set trg_definer: lex->definer->set_lex_string(trg_definer, trg_definer_holder); } else trg_definer= empty_lex_string; > + } > + > + LEX_STRING stmt_definition; > + stmt_definition.str= (char*) thd->lex->stmt_definition_begin; > + stmt_definition.length= thd->lex->stmt_definition_end - > + thd->lex->stmt_definition_begin; > + trim_whitespace(thd->charset(), & stmt_definition); > + stmt_query->append(stmt_definition.str, stmt_definition.length); > + return false; > +} > > /** > Create trigger for table. > @@ -639,16 +722,16 @@ bool Table_triggers_list::create_trigger(THD *thd, TABLE_LIST *tables, > char file_buff[FN_REFLEN], trigname_buff[FN_REFLEN]; > LEX_STRING file, trigname_file; > LEX_STRING *trg_def; > - LEX_STRING definer_user; > - LEX_STRING definer_host; > ulonglong *trg_sql_mode; > char trg_definer_holder[USER_HOST_BUFF_SIZE]; > - LEX_STRING *trg_definer; > + LEX_STRING *trg_definer= NULL; > Item_trigger_field *trg_field; > struct st_trigname trigname; > LEX_STRING *trg_client_cs_name; > LEX_STRING *trg_connection_cl_name; > LEX_STRING *trg_db_cl_name; > + sp_head *trg_head= bodies[lex->trg_chistics.event] > + [lex->trg_chistics.action_time]; trg_head as an element in a bodies array? May be trg_body instead? > > if (check_for_broken_triggers()) > return true; > diff --git a/sql/sql_view.cc b/sql/sql_view.cc > index a18193c..52b534e 100644 > --- a/sql/sql_view.cc > +++ b/sql/sql_view.cc > @@ -858,6 +863,14 @@ static int mysql_register_view(THD *thd, TABLE_LIST *view, > view->definer.host= lex->definer->host; > view->view_suid= lex->create_view_suid; > view->with_check= lex->create_view_check; > + > + DBUG_EXECUTE_IF("simulate_register_view_failure", > + my_error(ER_OUT_OF_RESOURCES, MYF(0));); > + DBUG_EXECUTE_IF("simulate_register_view_failure", > + error= -1;); > + DBUG_EXECUTE_IF("simulate_register_view_failure", > + goto err;); combine these in one DBUG_EXECUTE_IF: DBUG_EXECUTE_IF("simulate_register_view_failure", { my_error(ER_OUT_OF_RESOURCES, MYF(0)); error= -1; goto err; }); > + > if ((view->updatable_view= (can_be_merged && > view->algorithm != VIEW_ALGORITHM_TMPTABLE))) > { > diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy > index d6b3fa4..7da047f 100644 > --- a/sql/sql_yacc.yy > +++ b/sql/sql_yacc.yy > @@ -752,8 +752,18 @@ bool setup_select_in_parentheses(LEX *lex) > return FALSE; > } > > -static bool add_create_index_prepare (LEX *lex, Table_ident *table) > +static bool add_create_index_prepare (LEX *lex, Table_ident *table, > + uint create_or_replace, > + uint create_if_not_exists) > { > + if (create_or_replace && create_if_not_exists) > + { > + my_error(ER_WRONG_USAGE, MYF(0), "OR REPLACE", "IF NOT EXISTS"); > + return TRUE; > + } I'd rather see this check in the opt_if_not_exists rule, in only one place, that is. For this to work, though, the result of create_or_replace rule must be stored somewhere where opt_if_not_exists can access it. For example, in Lex->create_info.options. > + > + lex->key_create_info.options = (create_or_replace | create_if_not_exists); > + > lex->sql_command= SQLCOM_CREATE_INDEX; > if (!lex->current_select->add_table_to_list(lex->thd, table, NULL, > TL_OPTION_UPDATING, Regards, Sergei

1 0

[Maria-developers] MariaDB 10.0 on POWER
by Colin Charles 14 Oct '14

14 Oct '14

Hi, I'm sure you have read: https://www.flamingspork.com/blog/2014/10/08/mariadb-10-0-on-power/ For XL: There’s obviously still some work in progress going on, but my initial results show performance within around 10% of MySQL, so with a bit of work we will hopefully see MariaDB reach performance parity. For Serg: One interesting find was the code to account for thread memory usage uses a single atomic variable: this does not scale and does end up showing up on profiles. It isn't clear 10% loss to which MySQL, but comparing to 5.6 or 5.7, this has to "just work better" Thanks -- Colin Charles, Chief Evangelist, MariaDB Corporation blog: http://bytebot.net/blog/| t: +6-012-204-3201 | Skype: colincharles

2 1

Re: [Maria-developers] [Commits] 47fea67: MDEV-12 OpenGIS: create required tables: GeometryColumns, related views.
by Sergei Golubchik 14 Oct '14

14 Oct '14

Hi, Alexey! On Sep 02, Alexey Botchkov wrote: > revision-id: 47fea6798f0f37b8a42ad047d7daf6ec9bb52813 > parent(s): e44751b65f4760067d15f8a526e8f97f84810c29 > committer: Alexey Botchkov > branch nick: > timestamp: 2014-09-02 11:31:26 +0500 > message: > > MDEV-12 OpenGIS: create required tables: GeometryColumns, related views. > GEOMETRY_COLUMNS and SPATIAL_REF_SYS tables added to the INFORMATION_SCHEMA. > > diff --git a/mysql-test/r/information_schema.result > b/mysql-test/r/information_schema.result > index 6df5b89..93c9e90 100644 > --- a/mysql-test/r/information_schema.result > +++ b/mysql-test/r/information_schema.result > @@ -43,7 +43,8 @@ create view v1 (c) as > SELECT table_name FROM information_schema.TABLES > WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', > 'mysqltest') AND > table_name not like 'ndb_%' AND table_name not like 'innodb_%' AND > -table_name not like 'xtradb_%'; > +table_name not like 'xtradb_%' AND > +table_name not like 'geometry%' AND table_name not like 'spatial%'; Why did you filter these tables out? Normally, only plugin-created tables are filtered out, because these plugins might be disabled in the build. But we don't ever build without geometry, do we? I'd keep these tables in the result. > select * from v1; > c > ALL_PLUGINS > diff --git a/mysql-test/t/gis.test b/mysql-test/t/gis.test > index c387069..1a07a33 100644 > --- a/mysql-test/t/gis.test > +++ b/mysql-test/t/gis.test > @@ -1461,3 +1461,7 @@ insert into t1 values(Geomfromtext('POLYGON((1 1, > 2 2, 2 1, 1 1))')); > drop table t1; > > --echo End of 5.5 tests > + > +SHOW CREATE TABLE information_schema.geometry_columns; > +SHOW CREATE TABLE information_schema.spatial_ref_sys; Please, add tests for the content of these tables. Then ok to push (assuming these new tests will work fine). Regards, Sergei

1 0

[Maria-developers] Is it soon 'end of this year'?
by Peter Laursen 13 Oct '14

13 Oct '14

Around ½ year ago I discussed with some people here (Sergei G and/or Georg and/or Wlad) about support for Oracle-style plugins in the MariaDB C-API. The plugins include 1 Windows authentication plugin 2) Clear text plugin. At that time I was told that it could be considered 'by end end of this year'. 1) Windows authentication plugin. Not very important, but Sergei told that he had been playing wiht it and it could probably be done in ~1 day. 2) This is used by Oracle for setting up LDAP.authentication. I am very well aware that there is criticism of Oracle's implementation. But this does not change the fact hat people use it with servers from Oracle and expect clients to work with it. We get request(s) for it almost every week. So this is most important (for us). Any chance that those can soon be prioritized? (if you feel your self mis-quoted then I apologize).. -- Peter --Webyog

3 12

[Maria-developers] doubt about MDL
by Roberto Spadim 13 Oct '14

13 Oct '14

hi guys, i'm reading https://github.com/MariaDB/server/blob/10.1/sql/mdl.h i'm with a doubt... today we have the metadata lock plugin to check what mdl we have... my doubt is, how to expose how many threads are sharing the mdl key, and how many threads are waiting the mdl key, should it be possible or only one threrad could use one mdl? -- Roberto Spadim

2 3

Re: [Maria-developers] Page encryption exception handling.
by Sergei Golubchik 13 Oct '14

13 Oct '14

Hi, Clemens! On Oct 13, Clemens Dörrhöfer wrote: > Hi Sergei, > > I included an error, which was triggered by a missing key in the > keyfile. The server will see a wrong space ID, since it has not been > decrypted, which from my point of view is the reason for mariadb to > crash. > Basically we run into a problem when trying to decrypt a page using > fil_decrypt_page theoretically we can catch this exception much > sooner, since all we need to know is the key id we need for decryption > / encryption and throw an exception as soon as we know that we cannot > handle a key in best case before we trigger any aio. > > fil_decrypt_page() at fil0pageencryption.cc:391 0x555555feeb91 > os_file_read_func() at os0file.cc:3.156 0x555555e22648 > pfs_os_file_read_func() at os0file.ic:288 0x555555fdb4fa > fil_read_first_page() at fil0fil.cc:2.156 0x555555fe1eb0 > fil_validate_single_table_tablespace() at fil0fil.cc:4.303 0x555555fe5fd0 > fil_load_single_table_tablespace() at fil0fil.cc:4.443 0x555555fe64af > fil_load_single_table_tablespaces() at fil0fil.cc:4.806 0x555555fe6d1a > recv_init_crash_recovery() at log0recv.cc:3.046 0x555555e14447 > recv_recovery_from_checkpoint_start_func() at log0recv.cc:3.360 0x555555e14c69 > innobase_start_or_create_for_mysql() at srv0start.cc:2.471 0x555555eeea65 This still doesn't show where it crashes, only where you detect the missing key. But anyway, I agree, if you can detect a missing key sooner, you'd better do that, instead of returning invalid page to XtraDB, because it used to crash the server in case of severe errors like that. Regards, Sergei

1 0

[Maria-developers] Isn't thsi a bug?
by Peter Laursen 13 Oct '14

13 Oct '14

Refer: http://bugs.mysql.com/bug.php?id=74238 here I prefer the 'relaxed parsing' in Maria DB. But look at this: CREATE TABLE `d` ( `rec_id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT, `datetime` DATETIME(4) DEFAULT NULL, `timestamp` TIMESTAMP(4) NOT NULL DEFAULT CURRENT_TIMESTAMP(5) ON UPDATE CURRENT_TIMESTAMP(4), PRIMARY KEY (`rec_id`) ) ENGINE=INNODB; -- it actually creates the table SHOW CREATE TABLE d; /* returns CREATE TABLE `d` ( `rec_id` int(10) unsigned NOT NULL AUTO_INCREMENT, `datetime` datetime(4) DEFAULT NULL, `timestamp` timestamp(4) NOT NULL DEFAULT CURRENT_TIMESTAMP(4) ON UPDATE CURRENT_TIMESTAMP(4), PRIMARY KEY (`rec_id`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 */ I don't think DEFAULT CURRENT_TIMESTAMP(5) should be accepted for the datatype DATETIME(4). What say? -- Peter -- Webyog

2 2

[Maria-developers] Fwd: [JIRA] (MDEV-6566) Different INSERT behaviour on bad bytes with and without character set conversion
by Alexander Barkov 13 Oct '14

13 Oct '14

Hi Sergei, I suggest to fix this starting from 10.1. -------- Original Message -------- Subject: [JIRA] (MDEV-6566) Different INSERT behaviour on bad bytes with and without character set conversion Date: Thu, 9 Oct 2014 21:03:51 +0300 (EEST) From: Sergei Golubchik (JIRA) <jira(a)mariadb.atlassian.net> To: bar(a)mariadb.org [ https://mariadb.atlassian.net/browse/MDEV-6566?page=com.atlassian.jira.plug… ] Sergei Golubchik updated MDEV-6566: ----------------------------------- Fix Version/s: (was: 5.5.40) 5.5.41 > Different INSERT behaviour on bad bytes with and without character set conversion > --------------------------------------------------------------------------------- > > Key: MDEV-6566 > URL: https://mariadb.atlassian.net/browse/MDEV-6566 > Project: MariaDB Server > Issue Type: Bug > Affects Versions: 5.3.12, 5.5.39, 10.0.13 > Reporter: Alexander Barkov > Assignee: Alexander Barkov > Fix For: 10.0.15, 5.5.41 > > > If I run this SQL script in a utf8 client: > {noformat} > SET NAMES utf8; > DROP TABLE IF EXISTS t1; > CREATE TABLE t1 ( > a VARCHAR(10) CHARACTER SET ucs2, > b VARCHAR(10) CHARACTER SET utf8 > ); > INSERT INTO t1 SELECT 'a 😁 b', 'a 😁 b'; > SHOW WARNINGS; > SELECT * FROM t1; > {noformat} > It displays the following warnings: > {noformat} > +---------+------+----------------------------------------------------------------------+ > | Level | Code | Message | > +---------+------+----------------------------------------------------------------------+ > | Warning | 1300 | Invalid utf8 character string: '\xF0\x9F\x98\x81 b' | > | Warning | 1300 | Invalid utf8 character string: '\xF0\x9F\x98\x81 b' | > | Warning | 1366 | Incorrect string value: '\xF0\x9F\x98\x81 b' for column 'a' at row 1 | > | Warning | 1265 | Data truncated for column 'b' at row 1 | > +---------+------+----------------------------------------------------------------------+{noformat} > with the following results: > {noformat} > +----------+------+ > | a | b | > +----------+------+ > | a ???? b | a | > +----------+------+ > {noformat} > Notice, the character '😁' is "U+1F601 GRINNING FACE WITH SMILING EYES". It is outside of the BMP range supported by MariaDB character sets "utf8" and "ucs2". > The two columns worked differently: > - The ucs2 column correctly treated '😁' as an invalid sequence of four bytes, > converted it into four question marks, and appended the character 'b' after them. > - The utf8 column just stopped on this character and lost the trailing 'b'. > The warnings are also different for the two columns. > As "utf8" and "ucs2" are equitable character sets that support exactly the > same character range and repertoire (just with a different encoding), > it would be fair to expect the same behaviour of the two columns. > The column "a" exposes a better behaviour: it preserves as much > information as possible. > The column "b" should be fixed to work in the same way with "a". > The reason of the failure is that the conversion code that is activated > for the column "a" (with conversion) uses an mb_wc()..mb_wc() loop, while the > code activated for the column "b" (without conversion) copies only a well formed > prefix. > The code without conversion should not stop on a bad byte sequence > and try to copy as much data as possible, like the conversion code does. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003)

2 1

[Maria-developers] Page encryption exception handling.
by Clemens Dörrhöfer 13 Oct '14

13 Oct '14

Hi community, I am working on implementing page encryption for XtraDB. Encryption itself works fine, now I am focusing on exception handling. My problem is that whenever page encryption is misconfigured it will crash the entire server, which is no acceptable behaviour. Example I encrypted a table with key id 42, I delete the key from the keypool thus I can no longer decrypt the table, because the appropriate key is missing. Ideally whenever a client tries to query data from an encrypted table, which cannot be decrypted, it will receive an error message informing the client that Table X could not be decrypted because key 42 is missing. Can you point me out to how to accomplish such a behaviour. Greetings Clemens Dörrhöfer

2 1

Re: [Maria-developers] [Commits] [svoj@mariadb.org: Rev 4437: Fixed mroonga build failure on Power8: define generic gcc version of in lp:maria/10.0]
by Kouhei Sutou 09 Oct '14

09 Oct '14

Hi Sergey, Thanks for the fix! I've merged your change into Groonga: https://github.com/groonga/groonga/commit/b3d38c5b9863cb7a9e93b9461c8b93fbf… Thanks, -- kou In <20141009083146.GA3231@june> "[Commits] [svoj(a)mariadb.org: Rev 4437: Fixed mroonga build failure on Power8: define generic gcc version of in lp:maria/10.0]" on Thu, 9 Oct 2014 12:31:47 +0400, Sergey Vojtovich <svoj(a)mariadb.org> wrote: > Hi Kentoku, > > FYI: I just pushed this fix to 10.0. There're also some test failures on Power8 > in mroonga/storage suite, mroonga/wrapper was 100% successful. > > Regards, > Sergey > > ----- Forwarded message from Sergey Vojtovich <svoj(a)mariadb.org> ----- > > Date: Thu, 9 Oct 2014 08:25:58 +0000 (UTC) > From: Sergey Vojtovich <svoj(a)mariadb.org> > To: commits(a)mariadb.org > Subject: [Commits] Rev 4437: Fixed mroonga build failure on Power8: define generic gcc version of in lp:maria/10.0 > > At lp:maria/10.0 > > ------------------------------------------------------------ > revno: 4437 > revision-id: svoj(a)mariadb.org-20141009082545-nt183b7y1gjihfjp > parent: psergey(a)askmonty.org-20141006112922-45pcytb993xcwh89 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 10.0-mroonga > timestamp: Thu 2014-10-09 12:25:45 +0400 > message: > Fixed mroonga build failure on Power8: define generic gcc version of > GRN_SET_64BIT. > === modified file 'storage/mroonga/vendor/groonga/lib/groonga_in.h' > --- a/storage/mroonga/vendor/groonga/lib/groonga_in.h 2014-09-20 15:33:45 +0000 > +++ b/storage/mroonga/vendor/groonga/lib/groonga_in.h 2014-10-09 08:25:45 +0000 > @@ -505,6 +505,9 @@ typedef int grn_cond; > /* todo */ > # define GRN_SET_64BIT(p,v) \ > (void)atomic_swap_64(p, v) > +# elif defined(__ATOMIC_SEQ_CST) /* GCC atomic builtins */ > +# define GRN_SET_64BIT(p,v) \ > + __atomic_store_n(p, v, __ATOMIC_SEQ_CST) > # endif /* ATOMIC 64BIT SET */ > > # ifdef HAVE_MKOSTEMP > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits > > ----- End forwarded message ----- > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

2 2

[Maria-developers] INET6, data type plugins
by Alexander Barkov 09 Oct '14

09 Oct '14

Hello all, I have mostly implemented the server side part of the fixed length 16-byte INET6 data type (only some clean-ups left). MDEV-274 The data type for IPv6/IPv4 addresses in MariaDB https://mariadb.atlassian.net/browse/MDEV-274 I made it in a very new style, which is ready to data type plugins, as described in: MDEV-4912 Add a plugin to field types (column types) https://mariadb.atlassian.net/browse/MDEV-4912 It is done by introducing a new abstract super-class Type_handler with a number of virtual public methods (around 50 public methods at this point). To implement a new data type one just needs to implement an instantiable class derived from Type_handler and register the new data type in a special global registry. Everything start to work automatically, including (but not limited to): - correct comparison - hybrid type functions like COALESCE, LEAST - aggregate functions MAX() and MIN() - CAST to the new type - ORDER BY, both for field and for items - conversion to the standard data types - indexing and index search The definition of the instantiable class that actually implements INET6 looks like this: class Type_handler_inet6: public Type_handler { private: class Field_inet6: public Field { ... }; class Arg_comparator_inet6: public Arg_comparator { ... }; class Item_typecast_inet6: public Item_func { ... }; public: // various access methods }; It looks very nice for me so far :) I have some questions about the remaining things: 1. In my current patch the new data type INET6 is not loaded dynamically from a shared library. It's implemented inside mysqld. Is it Okey? Or would we prefer to make it as a plugin from the beginning? (in this case there's some more work left: the dynamic loader code, client-server protocol, PS protocol). 2. In case of pluggable data types we'll need some changes in the client-server protocol, to transfer data type names (and probably some more metadata) to the client side in some textual form. Any ideas on that? 3. PS protocol obviously needs two ways to bind data: in text format and in binary format. For example, in case of INET6 it would be nice to be able to bind parameters and fetch values using: - text representation "ffff::ffff" - and binary representation 0xFFFF000000000000000000000000FFFF. I can see some ways to do this: a. Every pluggable data type uses its own type code in MYSQL_BIND: char ip6[16]= "\xff\xff...\xff"; MYSQL_BIND bind; bind.buffer_type= 128; // This is INET6 bind.buffer= ip6; bind.length= sizeof(ip6); but the code (128) may vary between servers, depending on how the plugin is installed. b. Or we could introduce a singe new type code MYSQL_TYPE_RAW: char ip6[16]= "\xff\xff...\xff"; MYSQL_BIND bind; bind.buffer_type= MYSQL_TYPE_RAW; bind.buffer= ip6; bind.length= sizeof(ip6); The server should be able to treat a single type code with different data types without any problems, because it knows the parameter context. But this will not allow to mix values of different types though, for example insert an INET4 value into an INET6 column (or vice verse) won't be possible. c. Or we could support both "native" type codes and MYSQL_TYPE_RAW. 4. Do we need the INET4 data type (fixed length, 4 bytes)? 5. Do we need a PostgreSQL-alike INET data type which is able to store both INET4 and INET6 values (I guess this one should be variable length: 4-bytes for INET4 and 16 bytes for INET6 values. Any feedback would be appreciated. The current patch (attached) will be polished. but the general idea should already be clear from the current version. Thanks.

3 4

Re: [Maria-developers] [Commits] Rev 4391: MDEV-6676: Speculative parallel replication. in http://bazaar.launchpad.net/~maria-captains/maria/10.0
by Jonas Oreland 08 Oct '14

08 Oct '14

how about also adding a mode = 'auto' which currently can be == transaction, but in future could be extended to based on statistics flip actual modes back and forth ? /Jonas On Tue, Oct 7, 2014 at 4:20 PM, <knielsen(a)knielsen-hq.org> wrote: > At http://bazaar.launchpad.net/~maria-captains/maria/10.0 > > ------------------------------------------------------------ > revno: 4391 > revision-id: knielsen(a)knielsen-hq.org-20141007142037-y85xq099ltdiqno6 > parent: knielsen(a)knielsen-hq.org-20141003094323-nosx0y7vv0kjv20e > committer: Kristian Nielsen <knielsen(a)knielsen-hq.org> > branch nick: work-10.0-mdev6676 > timestamp: Tue 2014-10-07 16:20:37 +0200 > message: > MDEV-6676: Speculative parallel replication. > > Better syntax for configuring the parallel mode. Now it is a CHANGE > MASTER > option instead of a system variable, which makes it possible to configure > differently for different multi-source replication masters. > > Now also the domain-based replication mode can be disabled. > === modified file 'mysql-test/include/check-testcase.test' > --- a/mysql-test/include/check-testcase.test 2013-12-16 12:02:21 +0000 > +++ b/mysql-test/include/check-testcase.test 2014-10-07 14:20:37 +0000 > @@ -64,6 +64,7 @@ if ($tmp) > --echo Master_SSL_Crlpath # > --echo Using_Gtid No > --echo Gtid_IO_Pos # > + --echo Parallel_Mode domain,groupcommit > } > if (!$tmp) { > # Note: after WL#5177, fields 13-18 shall not be filtered-out. > > === modified file 'mysql-test/r/mysqld--help.result' > --- a/mysql-test/r/mysqld--help.result 2014-09-19 13:25:37 +0000 > +++ b/mysql-test/r/mysqld--help.result 2014-10-07 14:20:37 +0000 > @@ -915,17 +915,6 @@ > parallel replication thread when reading ahead in the > relay log looking for opportunities for parallel > replication. Only used when --slave-parallel-threads > 0. > - --slave-parallel-mode=name > - Controls what transactions are applied in parallel when > - using --slave-parallel-threads. Syntax: > - slave_paralle_mode=value[,value...], where "value" could > - be one or more of: "domain", to apply different > - replication domains in parallel; "groupcommit", to apply > - in parallel transactions that group-committed together on > - the master; "transactional", to optimistically try to > - apply all transactional DML in parallel; and "waiting" to > - extend "transactional" to even transactions that had to > - wait on the master. > --slave-parallel-threads=# > If non-zero, number of threads to spawn to apply in > parallel events on the slave that were group-committed on > @@ -1320,7 +1309,6 @@ slave-exec-mode STRICT > slave-max-allowed-packet 1073741824 > slave-net-timeout 3600 > slave-parallel-max-queued 131072 > -slave-parallel-mode domain,groupcommit > slave-parallel-threads 0 > slave-skip-errors (No default value) > slave-sql-verify-checksum TRUE > > === modified file 'mysql-test/suite/rpl/r/rpl_parallel.result' > --- a/mysql-test/suite/rpl/r/rpl_parallel.result 2014-08-15 > 09:31:13 +0000 > +++ b/mysql-test/suite/rpl/r/rpl_parallel.result 2014-10-07 > 14:20:37 +0000 > @@ -1,5 +1,5 @@ > -include/rpl_init.inc [topology=1->2] > -SET @old_parallel_threads=@@GLOBAL.slave_parallel_threads; > +include/master-slave.inc > +[connection master] > SET GLOBAL slave_parallel_threads=10; > ERROR HY000: This operation cannot be performed as you have a running > slave ''; run STOP SLAVE '' first > include/stop_slave.inc > @@ -923,8 +923,79 @@ SELECT * FROM t2 WHERE a >= 30 ORDER BY > 32 > 33 > 34 > +*** MDEV-6676 - test syntax of CHANGE MASTER TO PARALLEL_MODE=xxx *** > +Parallel_Mode = 'domain,groupcommit' > include/stop_slave.inc > -SET GLOBAL slave_parallel_threads=@old_parallel_threads; > +CHANGE MASTER TO parallel_mode=(,domain,groupcommit); > +ERROR 42000: You have an error in your SQL syntax; check the manual that > corresponds to your MariaDB server version for the right syntax to use near > 'domain,groupcommit)' at line 1 > +CHANGE MASTER TO parallel_mode=(domain,groupcommit,transactional); > +ERROR HY000: Invalid use of 'transactional' option for PARALLEL_MODE in > CHANGE MASTER TO > +CHANGE MASTER TO parallel_mode=(waiting,domain,transactional,waiting); > +ERROR HY000: Invalid use of 'waiting' option for PARALLEL_MODE in CHANGE > MASTER TO > +CHANGE MASTER TO parallel_mode=(domain,domain); > +ERROR HY000: Invalid use of 'domain' option for PARALLEL_MODE in CHANGE > MASTER TO > +CHANGE MASTER TO parallel_mode=(waiting,transactional,domain); > +Parallel_Mode = 'domain,transactional,waiting' > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > +Parallel_Mode = 'domain,groupcommit' > +*** MDEV-6676 - test that empty parallel_mode does not replicate in > parallel *** > +INSERT INTO t2 VALUES (40); > +include/save_master_gtid.inc > +CHANGE MASTER TO parallel_mode=(); > +SET @old_dbug= @@GLOBAL.debug_dbug; > +SET GLOBAL debug_dbug="+d,slave_crash_if_parallel_apply"; > +include/start_slave.inc > +include/sync_with_master_gtid.inc > +SELECT * FROM t2 WHERE a >= 40 ORDER BY a; > +a > +40 > +include/stop_slave.inc > +SET GLOBAL debug_dbug=@old_dbug; > +*** MDEV-6676 - test disabling domain-based parallel replication *** > +SET gtid_domain_id = 1; > +INSERT INTO t2 VALUES (41); > +INSERT INTO t2 VALUES (42); > +INSERT INTO t2 VALUES (43); > +INSERT INTO t2 VALUES (44); > +INSERT INTO t2 VALUES (45); > +INSERT INTO t2 VALUES (46); > +DELETE FROM t2 WHERE a >= 41; > +SET gtid_domain_id = 2; > +INSERT INTO t2 VALUES (41); > +INSERT INTO t2 VALUES (42); > +INSERT INTO t2 VALUES (43); > +INSERT INTO t2 VALUES (44); > +INSERT INTO t2 VALUES (45); > +INSERT INTO t2 VALUES (46); > +SET gtid_domain_id = 0; > +include/save_master_gtid.inc > +CHANGE MASTER TO parallel_mode=(groupcommit); > +include/start_slave.inc > +include/sync_with_master_gtid.inc > +SELECT * FROM t2 WHERE a >= 40 ORDER BY a; > +a > +40 > +41 > +42 > +43 > +44 > +45 > +46 > +include/stop_slave.inc > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > +include/start_slave.inc > +*** MDEV-6676 - test that parallel mode is saved correctly in master.info > across server restart *** > +Parallel_Mode = 'domain,groupcommit' > +include/stop_slave.inc > +CHANGE MASTER TO parallel_mode=(transactional,waiting); > +Parallel_Mode = 'transactional,waiting' > +include/start_slave.inc > +include/rpl_restart_server.inc [server_number=2] > +Parallel_Mode = 'transactional,waiting' > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > +include/start_slave.inc > +include/stop_slave.inc > +SET GLOBAL slave_parallel_threads=0; > include/start_slave.inc > SET DEBUG_SYNC= 'RESET'; > DROP function foo; > > === modified file 'mysql-test/suite/rpl/r/rpl_parallel_optimistic.result' > --- a/mysql-test/suite/rpl/r/rpl_parallel_optimistic.result 2014-09-24 > 19:25:10 +0000 > +++ b/mysql-test/suite/rpl/r/rpl_parallel_optimistic.result 2014-10-07 > 14:20:37 +0000 > @@ -2,11 +2,9 @@ include/rpl_init.inc [topology=1->2] > ALTER TABLE mysql.gtid_slave_pos ENGINE=InnoDB; > CREATE TABLE t1 (a int PRIMARY KEY, b INT) ENGINE=InnoDB; > SET @old_parallel_threads=@@GLOBAL.slave_parallel_threads; > -SET @old_mode=@@GLOBAL.slave_parallel_mode; > include/stop_slave.inc > SET GLOBAL slave_parallel_threads=10; > -SET GLOBAL slave_parallel_mode="domain,transactional"; > -CHANGE MASTER TO master_use_gtid=slave_pos; > +CHANGE MASTER TO master_use_gtid=slave_pos, > parallel_mode=(domain,transactional); > INSERT INTO t1 VALUES(1,1); > BEGIN; > INSERT INTO t1 VALUES(2,1); > @@ -288,8 +286,8 @@ SET GLOBAL binlog_format= @old_format; > SET GLOBAL tx_isolation= @old_isolation; > include/start_slave.inc > include/stop_slave.inc > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > SET GLOBAL slave_parallel_threads=@old_parallel_threads; > -SET GLOBAL slave_parallel_mode=@old_mode; > include/start_slave.inc > DROP TABLE t1, t2, t3; > include/rpl_end.inc > > === modified file > 'mysql-test/suite/rpl/r/rpl_parallel_optimistic_nobinlog.result' > --- a/mysql-test/suite/rpl/r/rpl_parallel_optimistic_nobinlog.result > 2014-09-24 19:25:10 +0000 > +++ b/mysql-test/suite/rpl/r/rpl_parallel_optimistic_nobinlog.result > 2014-10-07 14:20:37 +0000 > @@ -6,12 +6,10 @@ INSERT INTO t1 VALUES (1,0), (2,0), (3,0 > INSERT INTO t2 VALUES (1,0), (2,0); > SET @old_isolation= @@GLOBAL.tx_isolation; > SET @old_parallel_threads=@@GLOBAL.slave_parallel_threads; > -SET @old_mode=@@GLOBAL.slave_parallel_mode; > include/stop_slave.inc > SET GLOBAL TRANSACTION ISOLATION LEVEL READ COMMITTED; > SET GLOBAL slave_parallel_threads=10; > -SET GLOBAL slave_parallel_mode="domain,transactional,waiting"; > -CHANGE MASTER TO master_use_gtid=slave_pos; > +CHANGE MASTER TO master_use_gtid=slave_pos, > parallel_mode=(domain,transactional,waiting); > *** Test that we replicate correctly when using READ COMMITTED and > --log-slave-updates=0 on the slave *** > INSERT INTO t1 SELECT 4, COUNT(*) FROM t2; > INSERT INTO t2 SELECT 4, COUNT(*) FROM t1; > @@ -78,8 +76,8 @@ a b > 10 10 > include/stop_slave.inc > SET GLOBAL tx_isolation= @old_isolation; > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > SET GLOBAL slave_parallel_threads=@old_parallel_threads; > -SET GLOBAL slave_parallel_mode=@old_mode; > include/start_slave.inc > DROP TABLE t1, t2; > include/rpl_end.inc > > === modified file 'mysql-test/suite/rpl/t/rpl_parallel.test' > --- a/mysql-test/suite/rpl/t/rpl_parallel.test 2014-08-15 09:31:13 +0000 > +++ b/mysql-test/suite/rpl/t/rpl_parallel.test 2014-10-07 14:20:37 +0000 > @@ -1,13 +1,12 @@ > --source include/have_innodb.inc > --source include/have_debug.inc > --source include/have_debug_sync.inc > ---let $rpl_topology=1->2 > ---source include/rpl_init.inc > +--source include/master-slave.inc > > # Test various aspects of parallel replication. > > --connection server_2 > -SET @old_parallel_threads=@@GLOBAL.slave_parallel_threads; > +--let $old_parallel_threads=`SELECT @@GLOBAL.slave_parallel_threads` > --error ER_SLAVE_MUST_STOP > SET GLOBAL slave_parallel_threads=10; > --source include/stop_slave.inc > @@ -1466,9 +1465,100 @@ SET sql_slave_skip_counter= 1; > SELECT * FROM t2 WHERE a >= 30 ORDER BY a; > > > +--echo *** MDEV-6676 - test syntax of CHANGE MASTER TO PARALLEL_MODE=xxx > *** > +--connection server_2 > + > +--let $status_items= Parallel_Mode > +--source include/show_slave_status.inc > +--source include/stop_slave.inc > +--error ER_PARSE_ERROR > +CHANGE MASTER TO parallel_mode=(,domain,groupcommit); > +--error ER_INVALID_SLAVE_PARALLEL_MODE > +CHANGE MASTER TO parallel_mode=(domain,groupcommit,transactional); > +--error ER_INVALID_SLAVE_PARALLEL_MODE > +CHANGE MASTER TO parallel_mode=(waiting,domain,transactional,waiting); > +--error ER_INVALID_SLAVE_PARALLEL_MODE > +CHANGE MASTER TO parallel_mode=(domain,domain); > +CHANGE MASTER TO parallel_mode=(waiting,transactional,domain); > +--let $status_items= Parallel_Mode > +--source include/show_slave_status.inc > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > +--let $status_items= Parallel_Mode > +--source include/show_slave_status.inc > + > + > +--echo *** MDEV-6676 - test that empty parallel_mode does not replicate > in parallel *** > +--connection server_1 > +INSERT INTO t2 VALUES (40); > +--source include/save_master_gtid.inc > + > +--connection server_2 > +CHANGE MASTER TO parallel_mode=(); > +# Test that we do not use parallel apply, by injecting an unconditional > +# crash in the parallel apply code. > +SET @old_dbug= @@GLOBAL.debug_dbug; > +SET GLOBAL debug_dbug="+d,slave_crash_if_parallel_apply"; > +--source include/start_slave.inc > +--source include/sync_with_master_gtid.inc > +SELECT * FROM t2 WHERE a >= 40 ORDER BY a; > +--source include/stop_slave.inc > +SET GLOBAL debug_dbug=@old_dbug; > + > + > +--echo *** MDEV-6676 - test disabling domain-based parallel replication > *** > +--connection server_1 > +# Let's do a bunch of transactions that will conflict if run out-of-order > in > +# domain-based parallel replication mode. > +SET gtid_domain_id = 1; > +INSERT INTO t2 VALUES (41); > +INSERT INTO t2 VALUES (42); > +INSERT INTO t2 VALUES (43); > +INSERT INTO t2 VALUES (44); > +INSERT INTO t2 VALUES (45); > +INSERT INTO t2 VALUES (46); > +DELETE FROM t2 WHERE a >= 41; > +SET gtid_domain_id = 2; > +INSERT INTO t2 VALUES (41); > +INSERT INTO t2 VALUES (42); > +INSERT INTO t2 VALUES (43); > +INSERT INTO t2 VALUES (44); > +INSERT INTO t2 VALUES (45); > +INSERT INTO t2 VALUES (46); > +SET gtid_domain_id = 0; > +--source include/save_master_gtid.inc > +--connection server_2 > +CHANGE MASTER TO parallel_mode=(groupcommit); > +--source include/start_slave.inc > +--source include/sync_with_master_gtid.inc > +SELECT * FROM t2 WHERE a >= 40 ORDER BY a; > +--source include/stop_slave.inc > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > +--source include/start_slave.inc > + > + > +--echo *** MDEV-6676 - test that parallel mode is saved correctly in > master.info across server restart *** > +--connection server_2 > +--let $status_items= Parallel_Mode > +--source include/show_slave_status.inc > +--source include/stop_slave.inc > +CHANGE MASTER TO parallel_mode=(transactional,waiting); > +--let $status_items= Parallel_Mode > +--source include/show_slave_status.inc > +--source include/start_slave.inc > + > +--let $rpl_server_number= 2 > +--source include/rpl_restart_server.inc > + > +--connection server_2 > +--let $status_items= Parallel_Mode > +--source include/show_slave_status.inc > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > +--source include/start_slave.inc > + > + > --connection server_2 > --source include/stop_slave.inc > -SET GLOBAL slave_parallel_threads=@old_parallel_threads; > +eval SET GLOBAL slave_parallel_threads=$old_parallel_threads; > --source include/start_slave.inc > SET DEBUG_SYNC= 'RESET'; > > > === modified file 'mysql-test/suite/rpl/t/rpl_parallel_optimistic.test' > --- a/mysql-test/suite/rpl/t/rpl_parallel_optimistic.test 2014-09-24 > 19:25:10 +0000 > +++ b/mysql-test/suite/rpl/t/rpl_parallel_optimistic.test 2014-10-07 > 14:20:37 +0000 > @@ -11,11 +11,9 @@ CREATE TABLE t1 (a int PRIMARY KEY, b IN > --connection server_2 > --sync_with_master > SET @old_parallel_threads=@@GLOBAL.slave_parallel_threads; > -SET @old_mode=@@GLOBAL.slave_parallel_mode; > --source include/stop_slave.inc > SET GLOBAL slave_parallel_threads=10; > -SET GLOBAL slave_parallel_mode="domain,transactional"; > -CHANGE MASTER TO master_use_gtid=slave_pos; > +CHANGE MASTER TO master_use_gtid=slave_pos, > parallel_mode=(domain,transactional); > > > --connection server_1 > @@ -309,8 +307,8 @@ SET GLOBAL tx_isolation= @old_isolation; > > --connection server_2 > --source include/stop_slave.inc > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > SET GLOBAL slave_parallel_threads=@old_parallel_threads; > -SET GLOBAL slave_parallel_mode=@old_mode; > --source include/start_slave.inc > > --connection server_1 > > === modified file > 'mysql-test/suite/rpl/t/rpl_parallel_optimistic_nobinlog.test' > --- a/mysql-test/suite/rpl/t/rpl_parallel_optimistic_nobinlog.test > 2014-09-24 19:25:10 +0000 > +++ b/mysql-test/suite/rpl/t/rpl_parallel_optimistic_nobinlog.test > 2014-10-07 14:20:37 +0000 > @@ -16,12 +16,10 @@ INSERT INTO t2 VALUES (1,0), (2,0); > --sync_with_master > SET @old_isolation= @@GLOBAL.tx_isolation; > SET @old_parallel_threads=@@GLOBAL.slave_parallel_threads; > -SET @old_mode=@@GLOBAL.slave_parallel_mode; > --source include/stop_slave.inc > SET GLOBAL TRANSACTION ISOLATION LEVEL READ COMMITTED; > SET GLOBAL slave_parallel_threads=10; > -SET GLOBAL slave_parallel_mode="domain,transactional,waiting"; > -CHANGE MASTER TO master_use_gtid=slave_pos; > +CHANGE MASTER TO master_use_gtid=slave_pos, > parallel_mode=(domain,transactional,waiting); > > > --echo *** Test that we replicate correctly when using READ COMMITTED and > --log-slave-updates=0 on the slave *** > @@ -63,8 +61,8 @@ SELECT * FROM t2 ORDER BY a; > --connection server_2 > --source include/stop_slave.inc > SET GLOBAL tx_isolation= @old_isolation; > +CHANGE MASTER TO parallel_mode=(domain,groupcommit); > SET GLOBAL slave_parallel_threads=@old_parallel_threads; > -SET GLOBAL slave_parallel_mode=@old_mode; > --source include/start_slave.inc > > --connection server_1 > > === modified file 'sql/lex.h' > --- a/sql/lex.h 2014-02-02 09:00:36 +0000 > +++ b/sql/lex.h 2014-10-07 14:20:37 +0000 > @@ -194,6 +194,7 @@ static SYMBOL symbols[] = { > { "DISTINCTROW", SYM(DISTINCT)}, /* Access likes this */ > { "DIV", SYM(DIV_SYM)}, > { "DO", SYM(DO_SYM)}, > + { "DOMAIN", SYM(DOMAIN_SYM)}, > { "DOUBLE", SYM(DOUBLE_SYM)}, > { "DROP", SYM(DROP)}, > { "DUAL", SYM(DUAL_SYM)}, > @@ -257,6 +258,7 @@ static SYMBOL symbols[] = { > { "GRANT", SYM(GRANT)}, > { "GRANTS", SYM(GRANTS)}, > { "GROUP", SYM(GROUP_SYM)}, > + { "GROUPCOMMIT", SYM(GROUPCOMMIT_SYM)}, > { "HANDLER", SYM(HANDLER_SYM)}, > { "HARD", SYM(HARD_SYM)}, > { "HASH", SYM(HASH_SYM)}, > @@ -429,6 +431,7 @@ static SYMBOL symbols[] = { > { "PACK_KEYS", SYM(PACK_KEYS_SYM)}, > { "PAGE", SYM(PAGE_SYM)}, > { "PAGE_CHECKSUM", SYM(PAGE_CHECKSUM_SYM)}, > + { "PARALLEL_MODE", SYM(PARALLEL_MODE_SYM)}, > { "PARSER", SYM(PARSER_SYM)}, > { "PARSE_VCOL_EXPR", SYM(PARSE_VCOL_EXPR_SYM)}, > { "PARTIAL", SYM(PARTIAL)}, > @@ -650,6 +653,7 @@ static SYMBOL symbols[] = { > { "VIEW", SYM(VIEW_SYM)}, > { "VIRTUAL", SYM(VIRTUAL_SYM)}, > { "WAIT", SYM(WAIT_SYM)}, > + { "WAITING", SYM(WAITING_SYM)}, > { "WARNINGS", SYM(WARNINGS)}, > { "WEEK", SYM(WEEK_SYM)}, > { "WEIGHT_STRING", SYM(WEIGHT_STRING_SYM)}, > > === modified file 'sql/log_event.cc' > --- a/sql/log_event.cc 2014-09-25 13:47:58 +0000 > +++ b/sql/log_event.cc 2014-10-07 14:20:37 +0000 > @@ -12728,7 +12728,7 @@ bool rpl_get_position_info(const char ** > return FALSE; > #else > const Relay_log_info *rli= &(active_mi->rli); > - if (opt_slave_parallel_threads == 0) > + if (!rli->mi->using_parallel()) > { > *log_file_name= rli->group_master_log_name; > *log_pos= rli->group_master_log_pos + > > === modified file 'sql/mysqld.cc' > --- a/sql/mysqld.cc 2014-09-26 10:37:19 +0000 > +++ b/sql/mysqld.cc 2014-10-07 14:20:37 +0000 > @@ -562,8 +562,6 @@ ulong stored_program_cache_size= 0; > > ulong opt_slave_parallel_threads= 0; > ulong opt_slave_domain_parallel_threads= 0; > -ulonglong opt_slave_parallel_mode= > - SLAVE_PARALLEL_DOMAIN | SLAVE_PARALLEL_GROUPCOMMIT; > ulong opt_binlog_commit_wait_count= 0; > ulong opt_binlog_commit_wait_usec= 0; > ulong opt_slave_parallel_max_queued= 131072; > > === modified file 'sql/mysqld.h' > --- a/sql/mysqld.h 2014-09-26 10:37:19 +0000 > +++ b/sql/mysqld.h 2014-10-07 14:20:37 +0000 > @@ -187,7 +187,6 @@ extern ulong stored_program_cache_size; > extern ulong opt_slave_parallel_threads; > extern ulong opt_slave_domain_parallel_threads; > extern ulong opt_slave_parallel_max_queued; > -extern ulonglong opt_slave_parallel_mode; > extern ulong opt_binlog_commit_wait_count; > extern ulong opt_binlog_commit_wait_usec; > extern my_bool opt_gtid_ignore_duplicates; > > === modified file 'sql/rpl_mi.cc' > --- a/sql/rpl_mi.cc 2014-09-02 12:07:01 +0000 > +++ b/sql/rpl_mi.cc 2014-10-07 14:20:37 +0000 > @@ -40,7 +40,8 @@ Master_info::Master_info(LEX_STRING *con > heartbeat_period(0), received_heartbeats(0), master_id(0), > prev_master_id(0), > using_gtid(USE_GTID_NO), events_queued_since_last_gtid(0), > - gtid_reconnect_event_skip_count(0), gtid_event_seen(false) > + gtid_reconnect_event_skip_count(0), gtid_event_seen(false), > + parallel_mode(SLAVE_PARALLEL_DOMAIN|SLAVE_PARALLEL_GROUPCOMMIT) > { > host[0] = 0; user[0] = 0; password[0] = 0; > ssl_ca[0]= 0; ssl_capath[0]= 0; ssl_cert[0]= 0; > @@ -178,6 +179,7 @@ void init_master_log_pos(Master_info* mi > mi->events_queued_since_last_gtid= 0; > mi->gtid_reconnect_event_skip_count= 0; > mi->gtid_event_seen= false; > + mi->parallel_mode= SLAVE_PARALLEL_DOMAIN|SLAVE_PARALLEL_GROUPCOMMIT; > > /* Intentionally init ssl_verify_server_cert to 0, no option available > */ > mi->ssl_verify_server_cert= 0; > @@ -514,6 +516,22 @@ file '%s')", fname); > else > mi->using_gtid= Master_info::USE_GTID_NO; > } > + else if (!strncmp(buf, STRING_WITH_LEN("parallel_mode="))) > + { > + int val= atoi(buf + (sizeof("parallel_mode=") - 1)); > + mi->parallel_mode= val & (SLAVE_PARALLEL_DOMAIN | > + SLAVE_PARALLEL_GROUPCOMMIT | > + SLAVE_PARALLEL_TRX | > + SLAVE_PARALLEL_WAITING); > + } > + else if (!strncmp(buf, STRING_WITH_LEN("END_MARKER"))) > + { > + /* > + Guard agaist extra left-overs at the end of file, in case a > later > + update causes the file to shrink compared to earlier > contents. > + */ > + break; > + } > } > } > } > @@ -657,7 +675,9 @@ int flush_master_info(Master_info* mi, > my_b_printf(file, > > "%u\n%s\n%s\n%s\n%s\n%s\n%d\n%d\n%d\n%s\n%s\n%s\n%s\n%s\n%d\n%s\n%s\n%s\n%s\n%d\n%s\n%s\n" > "\n\n\n\n\n\n\n\n\n\n\n" > - "using_gtid=%d\n", > + "using_gtid=%d\n" > + "parallel_mode=%u\n" > + "END_MARKER\n", > LINES_IN_MASTER_INFO, > mi->master_log_name, llstr(mi->master_log_pos, lbuf), > mi->host, mi->user, > @@ -666,7 +686,7 @@ int flush_master_info(Master_info* mi, > mi->ssl_cipher, mi->ssl_key, mi->ssl_verify_server_cert, > heartbeat_buf, "", ignore_server_ids_buf, > "", 0, > - mi->ssl_crl, mi->ssl_crlpath, mi->using_gtid); > + mi->ssl_crl, mi->ssl_crlpath, mi->using_gtid, > mi->parallel_mode); > my_free(ignore_server_ids_buf); > err= flush_io_cache(file); > if (sync_masterinfo_period && !err && > > === modified file 'sql/rpl_mi.h' > --- a/sql/rpl_mi.h 2014-09-02 12:07:01 +0000 > +++ b/sql/rpl_mi.h 2014-10-07 14:20:37 +0000 > @@ -75,6 +75,10 @@ class Master_info : public Slave_reporti > return connection_name.str == 0; > } > static const char *using_gtid_astext(enum enum_using_gtid arg); > + bool using_parallel() > + { > + return opt_slave_parallel_threads > 0 && parallel_mode != 0; > + } > > /* the variables below are needed because we can change masters on the > fly */ > char master_log_name[FN_REFLEN+6]; /* Room for multi-*/ > @@ -178,6 +182,12 @@ class Master_info : public Slave_reporti > uint64 gtid_reconnect_event_skip_count; > /* gtid_event_seen is false until we receive first GTID event from > master. */ > bool gtid_event_seen; > + /* > + The parallel replication modes, if any. A combination (binary OR) of > any of > + SLAVE_PARALLEL_DOMAIN, SLAVE_PARALLEL_GROUPCOMMIT, > SLAVE_PARALLEL_TRX, and > + SLAVE_PARALLEL_WAITING. > + */ > + uint32 parallel_mode; > }; > int init_master_info(Master_info* mi, const char* master_info_fname, > const char* slave_info_fname, > > === modified file 'sql/rpl_parallel.cc' > --- a/sql/rpl_parallel.cc 2014-10-03 09:43:23 +0000 > +++ b/sql/rpl_parallel.cc 2014-10-07 14:20:37 +0000 > @@ -1891,6 +1891,7 @@ rpl_parallel::do_event(rpl_group_info *s > bool did_enter_cond= false; > PSI_stage_info old_stage; > > + DBUG_EXECUTE_IF("slave_crash_if_parallel_apply", DBUG_SUICIDE();); > /* Handle master log name change, seen in Rotate_log_event. */ > typ= ev->get_type_code(); > if (unlikely(typ == ROTATE_EVENT)) > @@ -1971,7 +1972,8 @@ rpl_parallel::do_event(rpl_group_info *s > if (typ == GTID_EVENT) > { > Gtid_log_event *gtid_ev= static_cast<Gtid_log_event *>(ev); > - uint32 domain_id= (rli->mi->using_gtid == Master_info::USE_GTID_NO ? > + uint32 domain_id= (rli->mi->using_gtid == Master_info::USE_GTID_NO || > + !(rli->mi->parallel_mode & SLAVE_PARALLEL_DOMAIN) ? > 0 : gtid_ev->domain_id); > if (!(e= find(domain_id))) > { > @@ -2012,7 +2014,7 @@ rpl_parallel::do_event(rpl_group_info *s > { > Gtid_log_event *gtid_ev= static_cast<Gtid_log_event *>(ev); > bool new_gco; > - ulonglong mode= opt_slave_parallel_mode; > + ulonglong mode= rli->mi->parallel_mode; > uchar gtid_flags= gtid_ev->flags2; > group_commit_orderer *gco; > uint8 force_switch_flag; > > === modified file 'sql/rpl_parallel.h' > --- a/sql/rpl_parallel.h 2014-10-03 09:43:23 +0000 > +++ b/sql/rpl_parallel.h 2014-10-07 14:20:37 +0000 > @@ -12,7 +12,11 @@ class Relay_log_info; > struct inuse_relaylog; > > > -/* Bit masks for the values in --slave-parallel-mode. */ > +/* > + Bit masks for the values in --slave-parallel-mode. > + Note that these values cannot be changed - they are stored in > master.info, > + so need to be possible to read back in a different version of the > server. > +*/ > #define SLAVE_PARALLEL_DOMAIN (1ULL << 0) > #define SLAVE_PARALLEL_GROUPCOMMIT (1ULL << 1) > #define SLAVE_PARALLEL_TRX (1ULL << 2) > > === modified file 'sql/share/errmsg-utf8.txt' > --- a/sql/share/errmsg-utf8.txt 2014-07-08 17:38:26 +0000 > +++ b/sql/share/errmsg-utf8.txt 2014-10-07 14:20:37 +0000 > @@ -7111,3 +7111,5 @@ ER_SLAVE_SKIP_NOT_IN_GTID > eng "When using GTID, @@sql_slave_skip_counter can not be used. > Instead, setting @@gtid_slave_pos explicitly can be used to skip to after a > given GTID position." > ER_TABLE_DEFINITION_TOO_BIG > eng "The definition for table %`s is too big" > +ER_INVALID_SLAVE_PARALLEL_MODE > + eng "Invalid use of '%s' option for PARALLEL_MODE in CHANGE > MASTER TO" > > === modified file 'sql/slave.cc' > --- a/sql/slave.cc 2014-09-02 12:07:01 +0000 > +++ b/sql/slave.cc 2014-10-07 14:20:37 +0000 > @@ -625,8 +625,7 @@ int terminate_slave_threads(Master_info* > if (thread_mask & (SLAVE_SQL|SLAVE_FORCE_ALL)) > { > DBUG_PRINT("info",("Terminating SQL thread")); > - if (opt_slave_parallel_threads > 0 && > - mi->rli.abort_slave && mi->rli.stop_for_until) > + if (mi->using_parallel() && mi->rli.abort_slave && > mi->rli.stop_for_until) > { > mi->rli.stop_for_until= false; > mi->rli.parallel.stop_during_until(); > @@ -2576,6 +2575,8 @@ static bool send_show_master_info_header > field_list.push_back(new Item_empty_string("Using_Gtid", > sizeof("Current_Pos")-1)); > field_list.push_back(new Item_empty_string("Gtid_IO_Pos", 30)); > + field_list.push_back(new Item_empty_string("Parallel_Mode", > + sizeof("domain,groupcommit,transactional,waiting")-1)); > if (full) > { > field_list.push_back(new Item_return_int("Retried_transactions", > @@ -2708,8 +2709,7 @@ static bool send_show_master_info_data(T > else > { > idle= mi->rli.sql_thread_caught_up; > - if (opt_slave_parallel_threads > 0 && idle && > - !mi->rli.parallel.workers_idle()) > + if (mi->using_parallel() && idle && > !mi->rli.parallel.workers_idle()) > idle= false; > } > if (idle) > @@ -2786,13 +2786,34 @@ static bool send_show_master_info_data(T > protocol->store(mi->ssl_ca, &my_charset_bin); > // Master_Ssl_Crlpath > protocol->store(mi->ssl_capath, &my_charset_bin); > + // Using_Gtid > protocol->store(mi->using_gtid_astext(mi->using_gtid), > &my_charset_bin); > + // Gtid_IO_Pos > { > char buff[30]; > String tmp(buff, sizeof(buff), system_charset_info); > mi->gtid_current_pos.to_string(&tmp); > protocol->store(tmp.ptr(), tmp.length(), &my_charset_bin); > } > + // Parallel_Mode > + { > + /* Note how sizeof("domain") has room for "domain," due to traling > 0. */ > + char buf[sizeof("domain") + sizeof("groupcommit") + > + sizeof("transactional") + sizeof("waiting") + 1]; > + char *p= buf; > + uint32 mode= mi->parallel_mode; > + if (mode & SLAVE_PARALLEL_DOMAIN) > + p= strmov(p, "domain,"); > + if (mode & SLAVE_PARALLEL_GROUPCOMMIT) > + p= strmov(p, "groupcommit,"); > + if (mode & SLAVE_PARALLEL_TRX) > + p= strmov(p, "transactional,"); > + if (mode & SLAVE_PARALLEL_WAITING) > + p= strmov(p, "waiting,"); > + if (p != buf) > + --p; // Discard last ',' > + protocol->store(buf, p-buf, &my_charset_bin); > + } > if (full) > { > protocol->store((uint32) mi->rli.retried_trans); > @@ -3521,7 +3542,7 @@ static int exec_relay_log_event(THD* thd > > update_state_of_relay_log(rli, ev); > > - if (opt_slave_parallel_threads > 0) > + if (rli->mi->using_parallel()) > { > int res= rli->parallel.do_event(serial_rgi, ev, event_size); > if (res >= 0) > @@ -4667,7 +4688,7 @@ log '%s' at position %s, relay log '%s' > } > } > > - if (opt_slave_parallel_threads > 0) > + if (mi->using_parallel()) > rli->parallel.wait_for_done(thd, rli); > > /* Thread stopped. Print the current replication position to the log */ > @@ -4693,7 +4714,7 @@ log '%s' at position %s, relay log '%s' > (We want the first one to be before the printout of stop position to > get the correct position printed.) > */ > - if (opt_slave_parallel_threads > 0) > + if (mi->using_parallel()) > rli->parallel.wait_for_done(thd, rli); > > /* > @@ -6314,7 +6335,7 @@ static Log_event* next_event(rpl_group_i > llstr(my_b_tell(cur_log),llbuf1), > llstr(rli->event_relay_log_pos,llbuf2))); > DBUG_ASSERT(my_b_tell(cur_log) >= BIN_LOG_HEADER_SIZE); > - DBUG_ASSERT(opt_slave_parallel_threads > 0 || > + DBUG_ASSERT(rli->mi->using_parallel() || > my_b_tell(cur_log) == rli->event_relay_log_pos); > } > #endif > > === modified file 'sql/sql_lex.h' > --- a/sql/sql_lex.h 2014-08-07 16:06:56 +0000 > +++ b/sql/sql_lex.h 2014-10-07 14:20:37 +0000 > @@ -218,11 +218,18 @@ struct LEX_MASTER_INFO > uint port, connect_retry; > float heartbeat_period; > /* > + Modes of parallel replication enabled, if any. A combination (binary > OR) of > + any of SLAVE_PARALLEL_DOMAIN, SLAVE_PARALLEL_GROUPCOMMIT, > + SLAVE_PARALLEL_TRX, and SLAVE_PARALLEL_WAITING. > + */ > + uint32 repl_parallel_mode; > + /* > Enum is used for making it possible to detect if the user > changed variable or if it should be left at old value > */ > enum {LEX_MI_UNCHANGED, LEX_MI_DISABLE, LEX_MI_ENABLE} > - ssl, ssl_verify_server_cert, heartbeat_opt, > repl_ignore_server_ids_opt; > + ssl, ssl_verify_server_cert, heartbeat_opt, > repl_ignore_server_ids_opt, > + repl_parallel_mode_opt; > enum { > LEX_GTID_UNCHANGED, LEX_GTID_NO, LEX_GTID_CURRENT_POS, > LEX_GTID_SLAVE_POS > } use_gtid_opt; > @@ -241,10 +248,11 @@ struct LEX_MASTER_INFO > pos= relay_log_pos= server_id= port= connect_retry= 0; > heartbeat_period= 0; > ssl= ssl_verify_server_cert= heartbeat_opt= > - repl_ignore_server_ids_opt= LEX_MI_UNCHANGED; > + repl_ignore_server_ids_opt= repl_parallel_mode_opt= > LEX_MI_UNCHANGED; > gtid_pos_str.length= 0; > gtid_pos_str.str= NULL; > use_gtid_opt= LEX_GTID_UNCHANGED; > + repl_parallel_mode= 0; > } > }; > > > === modified file 'sql/sql_repl.cc' > --- a/sql/sql_repl.cc 2014-08-07 16:06:56 +0000 > +++ b/sql/sql_repl.cc 2014-10-07 14:20:37 +0000 > @@ -3434,6 +3434,9 @@ bool change_master(THD* thd, Master_info > lex_mi->relay_log_name || lex_mi->relay_log_pos) > mi->using_gtid= Master_info::USE_GTID_NO; > > + if (lex_mi->repl_parallel_mode_opt != LEX_MASTER_INFO::LEX_MI_UNCHANGED) > + mi->parallel_mode= lex_mi->repl_parallel_mode; > + > /* > If user did specify neither host nor port nor any log name nor any log > pos, i.e. he specified only user/password/master_connect_retry, he > probably > > === modified file 'sql/sql_yacc.yy' > --- a/sql/sql_yacc.yy 2014-09-03 14:24:31 +0000 > +++ b/sql/sql_yacc.yy 2014-10-07 14:20:37 +0000 > @@ -1130,6 +1130,7 @@ bool my_yyoverflow(short **a, YYSTYPE ** > %token DIV_SYM > %token DOUBLE_SYM /* SQL-2003-R */ > %token DO_SYM > +%token DOMAIN_SYM > %token DROP /* SQL-2003-R */ > %token DUAL_SYM > %token DUMPFILE > @@ -1194,6 +1195,7 @@ bool my_yyoverflow(short **a, YYSTYPE ** > %token GRANT /* SQL-2003-R */ > %token GRANTS > %token GROUP_SYM /* SQL-2003-R */ > +%token GROUPCOMMIT_SYM > %token GROUP_CONCAT_SYM > %token GT_SYM /* OPERATOR */ > %token HANDLER_SYM > @@ -1376,6 +1378,7 @@ bool my_yyoverflow(short **a, YYSTYPE ** > %token PACK_KEYS_SYM > %token PAGE_SYM > %token PAGE_CHECKSUM_SYM > +%token PARALLEL_MODE_SYM > %token PARAM_MARKER > %token PARSER_SYM > %token PARSE_VCOL_EXPR_SYM > @@ -1604,6 +1607,7 @@ bool my_yyoverflow(short **a, YYSTYPE ** > %token VIEW_SYM /* SQL-2003-N */ > %token VIRTUAL_SYM > %token WAIT_SYM > +%token WAITING_SYM > %token WARNINGS > %token WEEK_SYM > %token WEIGHT_STRING_SYM > @@ -2244,6 +2248,14 @@ rule: <-- starts at col 1 > { > Lex->mi.repl_ignore_server_ids_opt= > LEX_MASTER_INFO::LEX_MI_ENABLE; > } > + | PARALLEL_MODE_SYM EQ '(' ')' > + { > + Lex->mi.repl_parallel_mode_opt= > LEX_MASTER_INFO::LEX_MI_DISABLE; > + } > + | PARALLEL_MODE_SYM EQ '(' parallel_mode_list ')' > + { > + Lex->mi.repl_parallel_mode_opt= > LEX_MASTER_INFO::LEX_MI_ENABLE; > + } > | > master_file_def > ; > @@ -2260,6 +2272,52 @@ rule: <-- starts at col 1 > insert_dynamic(&Lex->mi.repl_ignore_server_ids, (uchar*) > &($1)); > } > > +parallel_mode_list: > + parallel_mode_option > + | parallel_mode_list ',' parallel_mode_option > + ; > + > +parallel_mode_option: > + DOMAIN_SYM > + { > + if (Lex->mi.repl_parallel_mode & SLAVE_PARALLEL_DOMAIN) > + { > + my_error(ER_INVALID_SLAVE_PARALLEL_MODE, MYF(0), > "domain"); > + MYSQL_YYABORT; > + } > + Lex->mi.repl_parallel_mode|= SLAVE_PARALLEL_DOMAIN; > + } > + | GROUPCOMMIT_SYM > + { > + if (Lex->mi.repl_parallel_mode & > + (SLAVE_PARALLEL_GROUPCOMMIT|SLAVE_PARALLEL_TRX)) > + { > + my_error(ER_INVALID_SLAVE_PARALLEL_MODE, MYF(0), > "groupcommit"); > + MYSQL_YYABORT; > + } > + Lex->mi.repl_parallel_mode|= SLAVE_PARALLEL_GROUPCOMMIT; > + } > + | TRANSACTIONAL_SYM > + { > + if (Lex->mi.repl_parallel_mode & > + (SLAVE_PARALLEL_GROUPCOMMIT|SLAVE_PARALLEL_TRX)) > + { > + my_error(ER_INVALID_SLAVE_PARALLEL_MODE, MYF(0), > "transactional"); > + MYSQL_YYABORT; > + } > + Lex->mi.repl_parallel_mode|= SLAVE_PARALLEL_TRX; > + } > + | WAITING_SYM > + { > + if (Lex->mi.repl_parallel_mode & SLAVE_PARALLEL_WAITING) > + { > + my_error(ER_INVALID_SLAVE_PARALLEL_MODE, MYF(0), > "waiting"); > + MYSQL_YYABORT; > + } > + Lex->mi.repl_parallel_mode|= SLAVE_PARALLEL_WAITING; > + } > + > + > master_file_def: > MASTER_LOG_FILE_SYM EQ TEXT_STRING_sys > { > @@ -14098,6 +14156,7 @@ user: user_maybe_role > | DISABLE_SYM {} > | DISCARD {} > | DISK_SYM {} > + | DOMAIN_SYM {} > | DUMPFILE {} > | DUPLICATE_SYM {} > | DYNAMIC_SYM {} > @@ -14131,6 +14190,7 @@ user: user_maybe_role > | GET_FORMAT {} > | GRANTS {} > | GLOBAL_SYM {} > + | GROUPCOMMIT_SYM {} > | HASH_SYM {} > | HARD_SYM {} > | HOSTS_SYM {} > @@ -14221,6 +14281,7 @@ user: user_maybe_role > | ONLY_SYM {} > | PACK_KEYS_SYM {} > | PAGE_SYM {} > + | PARALLEL_MODE_SYM {} > | PARTIAL {} > | PARTITIONING_SYM {} > | PARTITIONS_SYM {} > @@ -14336,6 +14397,7 @@ user: user_maybe_role > | VALUE_SYM {} > | WARNINGS {} > | WAIT_SYM {} > + | WAITING_SYM {} > | WEEK_SYM {} > | WEIGHT_STRING_SYM {} > | WORK_SYM {} > > === modified file 'sql/sys_vars.cc' > --- a/sql/sys_vars.cc 2014-09-19 13:25:37 +0000 > +++ b/sql/sys_vars.cc 2014-10-07 14:20:37 +0000 > @@ -1834,31 +1834,11 @@ static Sys_var_ulong Sys_slave_parallel_ > VALID_RANGE(0,2147483647), DEFAULT(131072), BLOCK_SIZE(1)); > > > -static const char *slave_parallel_mode_names[] = { > - "domain", "groupcommit", "transactional", "waiting", NULL > -}; > - > -static Sys_var_set Sys_slave_parallel_mode( > - "slave_parallel_mode", > - "Controls what transactions are applied in parallel when using " > - "--slave-parallel-threads. Syntax: > slave_paralle_mode=value[,value...], " > - "where \"value\" could be one or more of: \"domain\", to apply " > - "different replication domains in parallel; \"groupcommit\", to > apply " > - "in parallel transactions that group-committed together on the > master; " > - "\"transactional\", to optimistically try to apply all > transactional " > - "DML in parallel; and \"waiting\" to extend \"transactional\" to " > - "even transactions that had to wait on the master.", > - GLOBAL_VAR(opt_slave_parallel_mode), CMD_LINE(REQUIRED_ARG), > - slave_parallel_mode_names, > - DEFAULT(SLAVE_PARALLEL_DOMAIN | > - SLAVE_PARALLEL_GROUPCOMMIT)); > - > - > static Sys_var_bit Sys_replicate_allow_parallel( > "replicate_allow_parallel", > "If set when a transaction is written to the binlog, that > transaction " > - "is allowed to replicate in parallel on a slave where " > - "slave_parallel_mode is set to \"transactional\". Can be cleared > for " > + "is allowed to replicate in parallel on a slave where > parallel_mode " > + "is set to \"transactional\" (in CHANGE MASTER). Can be cleared > for " > "transactions that are likely to cause a conflict if replicated in > " > "parallel, to avoid unnecessary rollback and retry.", > SESSION_ONLY(option_bits), NO_CMD_LINE, OPTION_RPL_ALLOW_PARALLEL, > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits >

2 1

[Maria-developers] MDEV-6429 (storage engine API for parallel replication) possibly a requirement for MDEV-6676 (speculative parallel replication)
by Kristian Nielsen 06 Oct '14

06 Oct '14

Hi Serg, Earlier this year we discussed MDEV-6429, which is to implement a proper storage engine API for a feature used in parallel replication. This is where InnoDB reports to the replication layer whenever transaction T1 needs to wait for T2; parallel replication uses this to detect and resolve deadlocks caused by enforced commit order. In 10.0, we decided to not do a "proper" API, instead just modify InnoDB to call into the server layer directly. In 10.0, the need for this API/feature is somewhat InnoDB-specific, it is not known whether any of the other transactional storage engines would need to do anything similar for parallel replication to work. If we implement MDEV-6676, however, then the need for the API becomes more critical. Because MDEV-6676 absolutely only works if we have a mechanism to detect conflicts that cause a deadlock with commit order, as was implemented for InnoDB. If we were to attempt to enable speculative parallel replication for eg. Tokutek transactions, without implementing the necessary check in Tokutek, then the slave would quickly end up hanging. Thinking more on this, I suppose what we really need is what you requested also during the original discussion: A true optional feature for storage engines to implement this additional check. And a way for the server layer to see if a storage engine has the implementation. If it does, then we can try to run its transactions in parallel as per MDEV-6676. If not, we will not try, and things will still work correctly (just without the parallelisation optimisation). Maybe for the first version, the best thing to do is to just hard-code a check for the InnoDB storage engine. This will allow us to get real-world testing to see if the new speculative parallel replication feature is interesting to any users. Then if it is, and there turns out to be interest from other storage engine venders (such as Tokutek) to also support speculative parallel replication, we can work with them to implement a proper API. This approach would also allow to consider the needs of a different storage engine when designing the API, which makes it much more likely that the API will be of general use and not too tied to InnoDB internals. What do you think? - Kristian.

1 0

[Maria-developers] MariaDB Debian packaging: pending merge to upstream
by Otto Kekäläinen 03 Oct '14

03 Oct '14

Hello! Now when the rehearsal is done using simple spelling error fix patches, I now posted the first big merge request about my 5.5/debian upon the upstream 5.5 branch. Issue at https://mariadb.atlassian.net/browse/MDEV-6284 Merge request at https://code.launchpad.net/~otto/maria/maria-fix-debpkg-5.5/+merge/221576 I also have these old merge request still open: https://code.launchpad.net/~otto/maria/maria-fix-debian-copyright-trunk/+me… https://code.launchpad.net/~otto/maria/maria-fix-debian-copyright-5.5/+merg… The sooner somebody has can merge these the less likely it is that there will be merge conflicts.. Thanks! -- Check out our blog at http://seravo.fi/blog and follow @ottokekalainen

2 6

Re: [Maria-developers] Fwd: [Commits] Rev 3897: bzr merge -r4123..4144 codership/5.6 in lp:~maria-captains/maria/maria-10.0-galera
by Philip Stoev 01 Oct '14

01 Oct '14

Hello, Thanks for taking the time to review this. Your concerns are spot-on. == modified file 'mysql-test/mysql-test-run.pl' --- a/mysql-test/mysql-test-run.pl 2014-09-29 00:43:56 +0000 +++ b/mysql-test/mysql-test-run.pl 2014-09-30 22:06:15 +0000 @@ -3189,6 +3189,9 @@ mtr_report(" - adding wsrep, galera to default test suites"); push @DEFAULT_SUITES, qw(wsrep galera); + # ADD scripts to $PATH to that wsrep_sst_* can be found + $ENV{'PATH'} = $ENV{'PATH'}.':'.$basedir.'/ scripts'; > Is this the right place to do this ? Serg already commented about the fact > that we should not "temper" mtr globally, instead only when galera tests > are run, I just wonder because at scripts directory contain other than > wsrep scripts, does this affect existing tests... Yes, you are right, I reverted this change and moved it into suite/galera/galera2nodes.cnf, which now looks like this: [ENV] PATH=/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:@ENV.MYSQL_BINDIR/scripts Unfortunately, in the context of .cnf files, MTR does not allow to simply append to the existing $PATH, so I have to reinitialize it from scratch. === modified file 'mysql-test/suite/galera/ galera_2nodes.cnf' --- a/mysql-test/suite/galera/galera_2nodes.cnf 2014-08-26 20:23:56 +0000 +++ b/mysql-test/suite/galera/galera_2nodes.cnf 2014-09-30 22:06:15 +0000 @@ -2,22 +2,36 @@ !include include/default_mysqld.cnf [mysqld.1] +innodb_file_per_table binlog-format=row +innodb_file_format=Barracuda +performance_schema=OFF + wsrep_provider=(a)ENV.WSREP_PROVIDER wsrep_cluster_address='gcomm://' wsrep_provider_options='base_port=(a)mysqld.1.#galera_port' wsrep_sst_receive_address='127.0.0.1:@mysqld.1.#sst_port' # enforce read-committed characteristics across the cluster wsrep_causal_reads=ON +wsrep_sync_wait = 7 +wsrep_node_address=127.0.0.1 +wsrep_node_incoming_address=127.0.0.1 > Why we need to have innodb_file_per_table=1, innodb_file_format=Barracuda > and performance_schema=OFF ? I have removed those options. performance_schema was OFF because there was an assertion that affected all the tests. Secondly, why wsrep_sync_wait=7 (can't these different numbers have more meaningful values i.e. enum ?) This is required to ensure that all statements see the latest state of the database, allowing for deterministic tests. With respect to using a bitmask vs. an enum, please feel free to open an issue in LaunchPad. Philip Stoev

1 0

Re: [Maria-developers] [Commits] Rev 3897: bzr merge -r4123..4144 codership/5.6 in lp:~maria-captains/maria/maria-10.0-galera
by Sergei Golubchik 01 Oct '14

01 Oct '14

Hi, Jan! On Oct 01, Jan Lindström wrote: > Hi Nirbhay, Philip > > First of all, it is good to see that we have more galera tests. Few > questions: > > == modified file 'mysql-test/mysql-test-run.pl' > --- a/mysql-test/mysql-test-run.pl 2014-09-29 00:43:56 +0000 > +++ b/mysql-test/mysql-test-run.pl 2014-09-30 22:06:15 +0000 > @@ -3189,6 +3189,9 @@ > mtr_report(" - adding wsrep, galera to default test suites"); > push @DEFAULT_SUITES, qw(wsrep galera); > > + # ADD scripts to $PATH to that wsrep_sst_* can be found > + $ENV{'PATH'} = $ENV{'PATH'}.':'.$basedir.'/scripts'; > > Is this the right place to do this ? Serg already commented about the fact > that we should not "temper" mtr globally, instead only when galera tests > are run, I just wonder because at scripts directory contain other than > wsrep scripts, does this affect existing tests... And, just to note, in 10.1 I did it somewhat differently: my ($path) = grep { -f "$_/wsrep_sst_rsync"; } "$::bindir/scripts", $::path_client_bindir; return "No SST scripts" unless $path; $ENV{PATH}="$path:$ENV{PATH}"; Note the differences: - I check whether a script really exists - I look in two locations - same paths where mtr looks for mysqlhotcopy - the path is added to the beginning of the PATH, because we want to find our scripts before any other similar scripts that might happen to be in the PATH because of the existing MariaDB-Galera installation Regards, Sergei

1 0

[Maria-developers] Aria bug with query cache - wrong result and wrong table invalidation
by Roberto Spadim 30 Sep '14

30 Sep '14

Hi guys, could someone take a look at MDEV-6817? i'm testing a stress test with aria and row_format=page the problem is, running many php>while(1){ mysql>select max(column) from table; php> get the value, sum+1 mysql>insert into table (the value of column+1) php>} i got some problems... the value of the first max(column) sometimes return an old value in others words, the query isn't consistent, or the table invalidation code is wrong i got a value 296 at query cache if i change the autocommit from 1 to 0, i got a value of 297, i could use SQL_NO_CACHE and got the same 297 value the problem only occur with row_format=page -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

Re: [Maria-developers] Patch: CREATE [[NO] FORCE] VIEW Links
by James Briggs 30 Sep '14

30 Sep '14

Hi folks. Thank you Elena for commenting on MDEV-6709 "Disambiguating "CREATE FORCE VIEW view1 AS SELECT * FROM TABLE1" "As I understand, your patch hasn't been accepted/merged into the main tree yet. Why don't you apply the bugfix in your tree and resubmit the patch?" 1) MDEV-6709 is a related but separate issue from my FORCE patch, applying to MDEV-356 and others. I think it should be a separate patch. 2) What would you like me to do next to get my original patch accepted? 3) Who is the current maintainer for views? I feel that the view features are only 50% implemented in MariaDB and need some design effort. Thanks, James Briggs. -- Cassandra/MySQL DBA. Available in San Jose area or remote. ________________________________ From: James Briggs <actionmessage(a)yahoo.com> To: Maria-developers <maria-developers(a)lists.launchpad.net> Sent: Monday, September 8, 2014 2:52 PM Subject: Patch: CREATE [[NO] FORCE] VIEW Links Hi List. I mailed two patches to the list to implement CREATE [[NO] FORCE] plus tests this weekend. It looks like the mailing list software filters out long posts, so here are links to the list archives: Patch: CREATE [[NO] FORCE] VIEW Options https://lists.launchpad.net/maria-developers/msg07681.html Patch: CREATE VIEW test program https://lists.launchpad.net/maria-developers/msg07682.htm Thanks, James Briggs. -- Available for MySQL/NoSQL DBA/Programming in San Jose area or remote.

1 0

[Maria-developers] [GSoC2014] Self-tuning cost coefficients
by Anshu Avinash 28 Sep '14

28 Sep '14

Hi all, I have written a documentation for framework which I have created for gathering and calculating cost coefficients for optimizer, as part of my GSoC project. It is available at http://igniting.in/gsoc2014/2014/08/20/documentation/. Any suggestions/reviews are welcome. Regards Anshu Avinash

3 3

[Maria-developers] MariaDB 10.0.14 got rid of replication safety warning without warning in release notes?
by Pavel Ivanov 26 Sep '14

26 Sep '14

Hi, I see that in MariaDB 10.0.14 Michael disabled replication safety warning for the case of INSERT ON DUPLICATE KEY UPDATE with several unique keys with a comment "We should assume that the store engine will report the first duplicate key for this case." http://bazaar.launchpad.net/~maria-captains/maria/10.0/revision/4400.1.3 What does that mean? Which key is "the first duplicate key" and why MariaDB relies on an engine behavior which AFAIK is not guaranteed? And why this serious change in behavior wasn't mentioned in the Release Notes? On the related note Release Notes mention the error log flood protection. Why it's not behind flag? How should I disable the feature and get back to the old behavior? Thank you, Pavel

2 2

Re: [Maria-developers] [Commits] 236cc6c: wsrep-related changes: removed some unnecessary files & minor modifications.
by Sergei Golubchik 26 Sep '14

26 Sep '14

Hi, Nirbhay! On Sep 25, nirbhay(a)skysql.com wrote: > revision-id: 236cc6cd49f250a43f0f6a794d5052207b2d9348 > parent(s): a756ac61900e28448776427e6ebbb0ef6f48d5cc > committer: Nirbhay Choubey > branch nick: server > timestamp: 2014-09-25 20:59:15 -0400 > message: > > wsrep-related changes: removed some unnecessary files & minor modifications. Nirbhay, I've ask your few times on irc and email *not* to do that. I already have all these changes in my tree, now I'll need to merge that. It adds no value, only wasted double work and wasted time for useless merges. Please don't do changes that I've already done! Regards, Sergei

1 0

Re: [Maria-developers] More upgrades needed for work-amd64-valgrind
by Sergey Petrunia 25 Sep '14

25 Sep '14

Hi Jani, On Thu, Sep 25, 2014 at 02:22:48PM +0300, Jani Tolonen wrote: > Hi Sergey, > > cmake version was just one minor version too old. I upgraded it and > libstdc++ and according to cmake documentation the error should now be > gone. Please let me know if there are still problems. It seems like not - valgrind builder has has built 10.1 tree and is now running tests (for the first time in ... :). Thank you for your help! BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] More upgrades needed for work-amd64-valgrind
by Sergey Petrunia 25 Sep '14

25 Sep '14

Hi Jani, Thanks. Please let me know when cmake upgrade is finished. On Wed, Sep 24, 2014 at 04:07:51PM +0300, Jani Tolonen wrote: > Hi Sergei, > > Ok, I will check if I can get cmake to work a bit later tonight. > > BR, > Jani > > On 24 September 2014 14:48, Sergey Petrunia <sergey(a)mariadb.com> wrote: > > > Hello Jani, > > > > Thanks for getting git to work for work-amd64-valgrind host. > > > > I am looking at > > http://buildbot.askmonty.org/buildbot/grid?branch=10.1 > > > > and see that "git update" step succeeds. However, the next step fails: > > > > > > http://buildbot.askmonty.org/buildbot/builders/work-amd64-valgrind/builds/6… > > > > it is probably because of cmake being too old. > > > > Could you please update cmake on work-amd64-valgrind, also? > > > > BR > > Sergei > > -- > > Sergei Petrunia, Software Developer > > MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog > > > > > > -- BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] BUG - MDEV 6781
by Roberto Spadim 25 Sep '14

25 Sep '14

Hi guys, i found a bug at query cache and sent a patch to solve it https://mariadb.atlassian.net/browse/MDEV-6781 https://github.com/rspadim/server/commit/5a5e47bf4141ebc3584ed365173dde87b7… there's a pull request at 10.1 github (that's only a line removing a +1 at database name length) please review the patch that was only found cause i'm using qc_info plugin from mdev-4682 thanks :) -- Roberto Spadim

1 0

[Maria-developers] OQGRAPH/others engine and query cache
by Roberto Spadim 24 Sep '14

24 Sep '14

Hi guys i was checking OQGRAPH limitations with query cache it can't use query cache cause we don't know when data base change, that's ok right? i think that's ok too some others engines have this 'problem' too the first answer is, since oqgraph can't know when data table changes, we can't cache oqgraph tables... i think that's right BUT... query cache 'save' table used in query cache, for example: select * from oq_graph this will include "select * from oq_graph" at query cache using table 'oq_graph' instead of only using 'oq_graph' we could include 'oq_graph' AND 'data_table' even when we 'only' use oq_graph at FROM part this is more interesting than turning query cache off to oq_graph, and this remove the problem of "the data table don't tell oqgraph when we need to invalidate query cache" and that's right! but since data_table changed it will invalidate all queries using it, including oq_graph queries =) check if i'm not wrong, i can try a patch with it if we don't have others query cache related problems (for example query cache don't support tables outside query) -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

[Maria-developers] More upgrades needed for work-amd64-valgrind
by Sergey Petrunia 24 Sep '14

24 Sep '14

Hello Jani, Thanks for getting git to work for work-amd64-valgrind host. I am looking at http://buildbot.askmonty.org/buildbot/grid?branch=10.1 and see that "git update" step succeeds. However, the next step fails: http://buildbot.askmonty.org/buildbot/builders/work-amd64-valgrind/builds/6… it is probably because of cmake being too old. Could you please update cmake on work-amd64-valgrind, also? BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Maria-discuss] MariaDB encryption
by Jonas Oreland 23 Sep '14

23 Sep '14

Hi again, > What is the type of license of your code? I asked internally about license, and it seems like we releasing dual gpl2/apache licensed code. > I would like to know, which interfaces from maria-DB you are using. I don't 100% understand the question. We didn't write any actual encryption code, but used the one provided in openssl. Other than that, we didn't really "use interfaces", but rather added/modified functionality/interfaces here and there. Can you be more specific ? /Jonas On Sat, Jun 7, 2014 at 11:20 PM, Elmar Eperiesi-Beck <elmar(a)eperiesi-beck.de > wrote: > Hi! > We (eperi) would be glad to do a joined work with Google. > Our solution works with MS-SQL, Oracle and other DBs and we are currently > porting it to MariaDB - and - as Monty said - its never to late to put some > sources together and make the best for the open source community. > > What is the type of license of your code? > > Jonas, I am looking forward to connect to you directly. > > Regards > Elmar > > Hi! > > > Hi Jonas, > > (same Jonas we know from NDBCLUSTER? :-) Good to see you again) > > > > On 6 Jun 2014, at 02:31, Jonas Oreland <jonaso(a)google.com> wrote: > > > >> Hi there, > >> I read this blog post > >> > http://monty-says.blogspot.com/2014/05/for-your-eyes-only-or-adding-better.… > >> and wanted to inform you that we at Google has developed > on-disk/block-level encryption for Innodb, aria (as used by temporary > tables), binlogs and temp-files. > >> The code is not yet published, but we expect it to be within a few > weeks or so. > >> We (of course?) think that it would be better if you instead of > developing new code > >> spent the time testing/reviewing ours. > > We are out course happy to do this! > > >> I'm happy to answer questions on the topic, > >> and will let you know once we've published it. > > The main question I have about the Innodb encryption is if it based on the > compression code we did for fusion-io? > The idea we had on our side was that by using the new compression hooks we > could add encryption with very little changes to the Innodb code. > Looking forward to when you are ready to publish the code so we can > discuss your changes in detail. > > > This is great news! > > > > From what I gather, from Monty's blog post (and a 1:1 we had some time > back), this is something done by a partner/external company that has a > mostly OSS solution, that we should integrate into 10.1 > > Yes, that's correct. It I would have known that Google was working on > encryption I would have included them in my discussions with eperi. > Fortunately it's not yet too late to do this. > I am sure eperi would like to work on the Google code as a base! > > > That said, Google's release of something that works for InnoDB, Aria, > binlogs, temp files (and presumably not too hard to add for MyISAM) is > something we should definitely review and target for 10.1 > > Yes! > > Regards, > Monty > > >

3 21

[Maria-developers] MDEV-4682
by Roberto Spadim 23 Sep '14

23 Sep '14

Hi guys, could check MDEV-4682? it's done to review thanks -- Roberto Spadim

2 2

[Maria-developers] Switching on use of components in Jira
by Rasmus Johansson 23 Sep '14

23 Sep '14

Hello all, I will later today switch on the usage of Components in Jira. Components are a way of grouping issues inside a project. The idea is to map every issue in Jira, being it a bug or a new feature to a component which describes to what area of MariaDB the issue belongs. For example a bug in Aria would be marked with the component "Storage Engine - Aria". Currently there are 37 components defined in Jira that should cover many areas, but I'm sure there are a few missing. There is one generic to which you can assign issues that doesn't fall into any other category, called OTHER. I'll include the list of components here in the end of this email. * What do you need to do? * The Component field will be a mandatory field when closing an issue. Therefore if a component (or several components) haven't been specified earlier for an issue, you'll have to choose at least one when closing an issue. The field has autocompletion so if you start typing for example "Galera" you'll be presented with the alternatives "Galera", "Galera Arbitrator garbd", and "Galera SST". If you don't find a suitable component then choose the component called "OTHER". * What is the benefit of using components? * It's a way of organizing issues inside a project, so that you easily can pull out all issues related to a certain area of MariaDB, e.g. if I wanted to know all bug fixes for replication I would click on the Replication component and get a list. There is the possibility to set up automatic assignees based on components and components can be used in filters and reports throughout Jira. I hope this won't be a burden. All feedback is of course welcome. BR, Rasmus * Components in the MariaDB Server project in JIRA * Data Definition - Alter Table Data Definition - Procedure Data Definition - Temporary Data Manipulation - Insert Delayed Data Manipulation - Subquery Dynamic Columns Events Full-text Search Galera Galera Arbitrator garbd Galera SST GIS Locale Settings OTHER Platform Debian Platform FreeBSD Platform Power Platform Windows Plugin - Audit Plugins Query Cache Replication Show Profile sql_error_log SSL Storage Engine - Aria Storage Engine - Cassandra Storage Engine - Connect Storage Engine - InnoDB Storage Engine - OQGRAPH Storage Engine - Spider Storage Engine - XtraDB Storage Enging - TokuDB Time zones Triggers Views XML Functions

3 4

[Maria-developers] Close some MDEV
by Roberto Spadim 22 Sep '14

22 Sep '14

Hi guys, could close these mdev: 1)MDEV-4599 (it will be implemented at mdev-4584) 2)MDEV-4581 (it will be implemented at mdev-4682) Please check this mdev-4682 too, i sent a patch via github, there's an option that should be implemented at compile time to turn stats on/off i didn't checked how to implement it yet, if anyone could help me :) it must set a macro #define QUERY_CACHE_QC_INFO_PLUGIN 1 at "configure" time, maybe via cmake gui or something like it maybe i should change QUERY_CACHE_QC_INFO_PLUGIN to QUERY_CACHE_INCLUDE_STATS to easier understand of macro definitions, help is wellcome here -- Roberto Spadim

1 0

Re: [Maria-developers] [Commits] Rev 4376: MDEV-6676: Speculative parallel replication in http://bazaar.launchpad.net/~maria-captains/maria/10.0
by Jonas Oreland 22 Sep '14

22 Sep '14

> > Intermediate commit. Patch is far from complete, but this small patch was > nevertheless sufficient to be able to sysbench-0.4 OLTP with full > parallelisation. > "full parallelisation" does that mean that X threads on master make slave achieve k*X higher throughput ? /Jonas

4 16

[Maria-developers] Need suggestions for benchmarking MDEV-6676 parallel replication
by Kristian Nielsen 22 Sep '14

22 Sep '14

Hi Axel, I'd like to ask your advice on some benchmarking of MDEV-6676. MDEV-6676 is about extending in-order parallel replication to be significantly more aggressive, and potentially get a _lot_ more parallelism on the slave. The basic idea is to just try to run _everything_ in parallel. If there is a conflict, the code will roll back the erroneous transaction and retry it, this time not in parallel. I have run a basic sysbench 0.5 OLTP on my laptop, and results were promising. However, I need something better than that. My main problem is that often when I run write-load benchmarks, the bottleneck is InnoDB log write or purge, or something like that. So the load does not really scale with multiple threads, even on the master, and parallel replication is not really that useful even if perfectly implemented. So do you have a suggestion for a write-load benchmark that scales well for medium concurrency, like 8-32 threads or something like that? And for a machine on which to run it? The best benchmark I think would be one that is I/O bound on a working set that is not in memory, on a machine with a disk subsystem that has good throughput at high concurrency. Because this should resemble the real-world use cases where parallel replication is most needed. Do you have any experience on running such benchmarks, and do we have any machines that could be used? Often the benchmarks I see in blogs are in-memory... Or do you have another suggestion? Thanks, - Kristian.

1 0

[Maria-developers] MDEV-4682 revision
by Roberto Spadim 20 Sep '14

20 Sep '14

hi guys, i sent some patch about qc_info plugin ( https://mariadb.atlassian.net/browse/MDEV-4682) via pull request at github (https://github.com/MariaDB/server/pull/6) could anyone check if it's ok? i included test file changes there's only one thing todo, that's include/not include statistics inside query cache, since it increase memory use, i think could be nice at runtime change this variable and include statistics on/off (more info inside mdev) -- Roberto Spadim

1 0

[Maria-developers] doubt about tests files
by Roberto Spadim 20 Sep '14

20 Sep '14

hi guys! i git cloned the last verison of mariadb from github, but i didn't found test files is it outside mariadb github source? -- Roberto Spadim

1 1

Re: [Maria-developers] [Commits] c8496e4: %B
by Roberto Spadim 19 Sep '14

19 Sep '14

working =P 2014-09-19 18:04 GMT-03:00 Igor Babaev <igor(a)askmonty.org>: > revision-id: c8496e48c4b7c35e9702f98daa19cdb4ef190097 > parent(s): 0486b3b796ad7ec597bff775da248096731c58d5 > committer: Igor Babaev > branch nick: 10.1-test-commit > timestamp: 2014-09-19 13:47:43 -0700 > message: > > %B > --- > sql/opt_range.cc | 2 ++ > 1 files changed, 2 insertions(+), 0 deletions(-) > > diff --git a/sql/opt_range.cc b/sql/opt_range.cc > index 27e20c0..c35bde6 100644 > --- a/sql/opt_range.cc > +++ b/sql/opt_range.cc > @@ -14,6 +14,8 @@ > along with this program; if not, write to the Free Software > Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 > USA */ > // test commit in git > +test 2 > + > /* > TODO: > Fix that MAYBE_KEY are stored in the tree so that we can detect use > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits > -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

[Maria-developers] Free Software Foundation address
by Matej Mužila 17 Sep '14

17 Sep '14

Hi, the address of the Free Software Foundation has changed and I think itshould be changed in sources too. Nowdays the correct one is: Free Software Foundation 51 Franklin Street, Fifth Floor Boston, MA 02110-1301 USA http://www.fsf.org/about/contact/ Regards Matej Mužila

1 0

[Maria-developers] master.info and relay-log.info location
by Tim Callaghan 15 Sep '14

15 Sep '14

Has any work been done in MariaDB 10.0 or 10.1 to move them from their file-based location to within XtraDB (and thus potentially TokuDB)? The knowledge base article at https://mariadb.com/kb/en/mariadb/development/general-development-informati… lists the following as a roadmap feature: - Transactional storage of slave state, rather than file-based master.info and relay-log.info . So the slave can recover consistently after a crash.

2 1

[Maria-developers] contribution procedure
by Jonas Oreland 12 Sep '14

12 Sep '14

Hi Serg, hope all is well. I created https://mariadb.atlassian.net/browse/MDEV-6712 for a plugin feature I wrote, and now I humbly ponder on how I get this review/triaged/something. What would be your best advice? /Jonas

2 1

[Maria-developers] How to get all fields in a given index?
by Rongrong 10 Sep '14

10 Sep '14

Hi, I'm trying to implement some feature that requires to get the list of fields for a given index. I'm trying to follow the optimizer code to find some clue but am not successful so far. Can anyone give me some pointers? Thanks! Rongrong

2 2

[Maria-developers] Total breakdown of development procedures :-(
by Kristian Nielsen 10 Sep '14

10 Sep '14

Today, I learned that a large change (the galera merge) has been pushed into 10.1 on August 27. At that point, there was not a _single_ builder in buildbot that was green! As a result, we now have an extremely serious regression in the server caused by this push, easily seen in the testsuite. This is a disaster! Someone (or more likely everyone) have forgotten the basic rules, rules that have been in place since the beginning of MariaDB, and which are essential to ensuring the necessary quality of a product used in production-critical environments every day: - No pushes into a tree with known test failures in Buildbot. - No pushes of larger changes without first seeing them green in Buildbot in a feature tree. - No pushes without review. - No pushes of new features that are not of beta quality. Must be feature complete, must have passed tests and review, must have documentation ready. I have asked Jan to revert the Galera merge until it can be tested green in Buildbot and passed RQG replication tests by Elena. But the problem here is clearly larger than any single patch. Things have degenerated far beyond what is even remotely acceptable. We need to step back and start following the rules to the letter to recover from this. - Kristian.

1 0

Re: [Maria-developers] [Commits] Rev 4385: MDEV-6459 max_relay_log_size and sql_slave_skip_counter misbehave on PPC64 in lp:~maria-captains/maria/10.0
by Sergey Vojtovich 09 Sep '14

09 Sep '14

Hi Sergei, just curious: any special reason to make these variables ulonglong? Both seem to fit into uint. Thanks, Sergey On Tue, Sep 09, 2014 at 04:52:12PM +0200, Sergei Golubchik wrote: > At lp:~maria-captains/maria/10.0 > > ------------------------------------------------------------ > revno: 4385 > revision-id: sergii(a)pisem.net-20140909145212-62txhaq57k301s16 > parent: sergii(a)pisem.net-20140909130235-h2sy95zwu55701l0 > fixes bug: https://mariadb.atlassian.net/browse/MDEV-6459 > committer: Sergei Golubchik <sergii(a)pisem.net> > branch nick: 10.0 > timestamp: Tue 2014-09-09 16:52:12 +0200 > message: > MDEV-6459 max_relay_log_size and sql_slave_skip_counter misbehave on PPC64 > > make slave_skip_counter and max_relay_log_size ulonglong > (sysvars should generally never be ulong) > === modified file 'sql/rpl_rli.h' > --- a/sql/rpl_rli.h 2014-09-08 10:59:57 +0000 > +++ b/sql/rpl_rli.h 2014-09-09 14:52:12 +0000 > @@ -251,10 +251,11 @@ class Relay_log_info : public Slave_repo > errors, and have been manually applied by DBA already. > Must be ulong as it's refered to from set_var.cc > */ > - volatile ulong slave_skip_counter; > + volatile ulonglong slave_skip_counter; > + ulonglong max_relay_log_size; > + > volatile ulong abort_pos_wait; /* Incremented on change master */ > volatile ulong slave_run_id; /* Incremented on slave start */ > - ulong max_relay_log_size; > mysql_mutex_t log_space_lock; > mysql_cond_t log_space_cond; > /* > > === modified file 'sql/sql_class.h' > --- a/sql/sql_class.h 2014-07-31 09:08:56 +0000 > +++ b/sql/sql_class.h 2014-09-09 14:52:12 +0000 > @@ -519,6 +519,14 @@ typedef struct system_variables > ulonglong sortbuff_size; > ulonglong group_concat_max_len; > ulonglong default_regex_flags; > + > + /** > + Place holders to store Multi-source variables in sys_var.cc during > + update and show of variables. > + */ > + ulonglong slave_skip_counter; > + ulonglong max_relay_log_size; > + > ha_rows select_limit; > ha_rows max_join_size; > ha_rows expensive_subquery_limit; > @@ -587,12 +595,6 @@ typedef struct system_variables > */ > uint32 gtid_domain_id; > uint64 gtid_seq_no; > - /** > - Place holders to store Multi-source variables in sys_var.cc during > - update and show of variables. > - */ > - ulong slave_skip_counter; > - ulong max_relay_log_size; > > /** > Default transaction access mode. READ ONLY (true) or READ WRITE (false). > > === modified file 'sql/sys_vars.cc' > --- a/sql/sys_vars.cc 2014-08-07 16:06:56 +0000 > +++ b/sql/sys_vars.cc 2014-09-09 14:52:12 +0000 > @@ -4248,11 +4248,11 @@ static Sys_var_uint Sys_slave_net_timeou > Return 0 + warning if it doesn't exist > */ > > -ulong Sys_var_multi_source_ulong:: > -get_master_info_ulong_value(THD *thd, ptrdiff_t offset) > +ulonglong Sys_var_multi_source_ulonglong:: > +get_master_info_ulonglong_value(THD *thd, ptrdiff_t offset) > { > Master_info *mi; > - ulong res= 0; // Default value > + ulonglong res= 0; // Default value > mysql_mutex_unlock(&LOCK_global_system_variables); > mysql_mutex_lock(&LOCK_active_mi); > mi= master_info_index-> > @@ -4261,7 +4261,7 @@ get_master_info_ulong_value(THD *thd, pt > if (mi) > { > mysql_mutex_lock(&mi->rli.data_lock); > - res= *((ulong*) (((uchar*) mi) + master_info_offset)); > + res= *((ulonglong*) (((uchar*) mi) + master_info_offset)); > mysql_mutex_unlock(&mi->rli.data_lock); > } > mysql_mutex_unlock(&LOCK_active_mi); > @@ -4273,7 +4273,7 @@ get_master_info_ulong_value(THD *thd, pt > bool update_multi_source_variable(sys_var *self_var, THD *thd, > enum_var_type type) > { > - Sys_var_multi_source_ulong *self= (Sys_var_multi_source_ulong*) self_var; > + Sys_var_multi_source_ulonglong *self= (Sys_var_multi_source_ulonglong*) self_var; > bool result= true; > Master_info *mi; > > @@ -4310,16 +4310,12 @@ static bool update_slave_skip_counter(sy > return false; > } > > - > -static Sys_var_multi_source_ulong > -Sys_slave_skip_counter("sql_slave_skip_counter", > - "Skip the next N events from the master log", > - SESSION_VAR(slave_skip_counter), > - NO_CMD_LINE, > - my_offsetof(Master_info, rli.slave_skip_counter), > - VALID_RANGE(0, UINT_MAX), DEFAULT(0), BLOCK_SIZE(1), > - ON_UPDATE(update_slave_skip_counter)); > - > +static Sys_var_multi_source_ulonglong Sys_slave_skip_counter( > + "sql_slave_skip_counter", "Skip the next N events from the master log", > + SESSION_VAR(slave_skip_counter), NO_CMD_LINE, > + MASTER_INFO_VAR(rli.slave_skip_counter), > + VALID_RANGE(0, UINT_MAX), DEFAULT(0), BLOCK_SIZE(1), > + ON_UPDATE(update_slave_skip_counter)); > > static bool update_max_relay_log_size(sys_var *self, THD *thd, Master_info *mi) > { > @@ -4328,17 +4324,14 @@ static bool update_max_relay_log_size(sy > return false; > } > > -static Sys_var_multi_source_ulong > -Sys_max_relay_log_size( "max_relay_log_size", > - "relay log will be rotated automatically when the " > - "size exceeds this value. If 0 are startup, it's " > - "set to max_binlog_size", > - SESSION_VAR(max_relay_log_size), > - CMD_LINE(REQUIRED_ARG), > - my_offsetof(Master_info, rli.max_relay_log_size), > - VALID_RANGE(0, 1024L*1024*1024), DEFAULT(0), > - BLOCK_SIZE(IO_SIZE), > - ON_UPDATE(update_max_relay_log_size)); > +static Sys_var_multi_source_ulonglong Sys_max_relay_log_size( > + "max_relay_log_size", > + "relay log will be rotated automatically when the size exceeds this " > + "value. If 0 are startup, it's set to max_binlog_size", > + SESSION_VAR(max_relay_log_size), CMD_LINE(REQUIRED_ARG), > + MASTER_INFO_VAR(rli.max_relay_log_size), > + VALID_RANGE(0, 1024L*1024*1024), DEFAULT(0), BLOCK_SIZE(IO_SIZE), > + ON_UPDATE(update_max_relay_log_size)); > > static Sys_var_charptr Sys_slave_skip_errors( > "slave_skip_errors", "Tells the slave thread to continue " > > === modified file 'sql/sys_vars.h' > --- a/sql/sys_vars.h 2014-07-18 15:45:21 +0000 > +++ b/sql/sys_vars.h 2014-09-09 14:52:12 +0000 > @@ -1993,7 +1993,8 @@ class Sys_var_replicate_events_marked_fo > like sql_slave_skip_counter are GLOBAL. > */ > > -class Sys_var_multi_source_ulong; > +#define MASTER_INFO_VAR(X) my_offsetof(Master_info, X), sizeof(Master_info::X) > +class Sys_var_multi_source_ulonglong; > class Master_info; > > typedef bool (*on_multi_source_update_function)(sys_var *self, THD *thd, > @@ -2002,31 +2003,27 @@ bool update_multi_source_variable(sys_va > THD *thd, enum_var_type type); > > > -class Sys_var_multi_source_ulong :public Sys_var_ulong > +class Sys_var_multi_source_ulonglong :public Sys_var_ulonglong > { > ptrdiff_t master_info_offset; > on_multi_source_update_function update_multi_source_variable_func; > public: > - Sys_var_multi_source_ulong(const char *name_arg, > + Sys_var_multi_source_ulonglong(const char *name_arg, > const char *comment, int flag_args, > ptrdiff_t off, size_t size, > CMD_LINE getopt, > ptrdiff_t master_info_offset_arg, > - ulong min_val, ulong max_val, ulong def_val, > - uint block_size, > + size_t master_info_arg_size, > + ulonglong min_val, ulonglong max_val, > + ulonglong def_val, uint block_size, > on_multi_source_update_function on_update_func) > - :Sys_var_ulong(name_arg, comment, flag_args, off, size, > - getopt, min_val, max_val, def_val, block_size, > - 0, VARIABLE_NOT_IN_BINLOG, 0, update_multi_source_variable), > + :Sys_var_ulonglong(name_arg, comment, flag_args, off, size, > + getopt, min_val, max_val, def_val, block_size, > + 0, VARIABLE_NOT_IN_BINLOG, 0, update_multi_source_variable), > master_info_offset(master_info_offset_arg), > update_multi_source_variable_func(on_update_func) > { > - } > - bool session_update(THD *thd, set_var *var) > - { > - session_var(thd, ulong)= (ulong) (var->save_result.ulonglong_value); > - /* Value should be moved to multi_master in on_update_func */ > - return false; > + SYSVAR_ASSERT(master_info_arg_size == size); > } > bool global_update(THD *thd, set_var *var) > { > @@ -2039,9 +2036,9 @@ class Sys_var_multi_source_ulong :public > } > uchar *session_value_ptr(THD *thd,LEX_STRING *base) > { > - ulong *tmp, res; > - tmp= (ulong*) (((uchar*)&(thd->variables)) + offset); > - res= get_master_info_ulong_value(thd, master_info_offset); > + ulonglong *tmp, res; > + tmp= (ulonglong*) (((uchar*)&(thd->variables)) + offset); > + res= get_master_info_ulonglong_value(thd, master_info_offset); > *tmp= res; > return (uchar*) tmp; > } > @@ -2049,7 +2046,7 @@ class Sys_var_multi_source_ulong :public > { > return session_value_ptr(thd, base); > } > - ulong get_master_info_ulong_value(THD *thd, ptrdiff_t offset); > + ulonglong get_master_info_ulonglong_value(THD *thd, ptrdiff_t offset); > bool update_variable(THD *thd, Master_info *mi) > { > return update_multi_source_variable_func(this, thd, mi); > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

2 1

[Maria-developers] Patch: CREATE VIEW test program
by James Briggs 08 Sep '14

08 Sep '14

Hi List. Below is tests/view.pl for testing CREATE VIEW options. It tests several thousand permutations and was used to find MDEV-6709 Thanks, James Briggs. -- Available for MySQL/NoSQL DBA/Programming in San Jose area or remote. tests/view.pl: #!/usr/bin/perl # Program: view.pl # Licence: GPL version 2 # Author: James Briggs # Date: 2014 09 07 # Env: Perl 5 # Note: requires CREATE [[NO] FORCE] option # # Testing of views. # use strict; use warnings; use DBI; use Getopt::Long; $|=1; use vars qw($opt_help $opt_Information $opt_force $opt_debug $opt_verbose $opt_root_user $opt_root_password $opt_user $opt_password $opt_database $opt_host $opt_silent); $opt_help = $opt_Information = $opt_force = $opt_debug = $opt_verbose = $opt_silent = 0; $opt_host = "localhost", $opt_root_user = "root"; $opt_root_password = ""; $opt_user = "view_user"; $opt_password = "view_user"; $opt_database = "view_test"; $opt_force = 1; my $version = "1.0"; my $opt_table="table1"; my $opt_user2 = $opt_user . '2'; GetOptions("Information","help","server=s","root-user=s","root-password=s","user","password=s","database=s","force","host=s","debug","verbose","silent") || usage(); usage() if ($opt_help || $opt_Information); # magic constants use constant N_TEST_RECORDS => 3; use constant HAS_VIEW_GRANTS => 0; use constant NO_VIEW_GRANTS => 1; my @cols = qw[id col1 col2 col3 col4]; my $tmp_table="/tmp/mysql-view.test"; unlink($tmp_table); # %roles array offets use constant U_DBH => 0; use constant U_PRIV_LEVEL => 1; use constant U_USER => 2; use constant U_HOST => 3; use constant U_PASSWORD => 4; # run tests with multiple user accounts from SUPER to very basic privs my %roles = ( # [ U_DBH, U_PRIV_LEVEL, U_USER, U_HOST, U_PASSWORD ] root => [ undef, HAS_VIEW_GRANTS, $opt_root_user, $opt_host, $opt_root_password ], power => [ undef, HAS_VIEW_GRANTS, $opt_user, $opt_host, $opt_password ], crud => [ undef, NO_VIEW_GRANTS, $opt_user2, $opt_host, $opt_password], # use the non-root user account again, this time with less privs ); if (!$opt_force) { print_info(); } # # setup test database # my $dbh = user_connect($opt_root_user,$opt_root_password, 0, 'test'); # $opt_database may not exist yet, so connect with 'test' $roles{'root'}->[U_DBH] = $dbh; test_query('root', "drop database if exists $opt_database"); # drop database to quickly drop any tables and views test_query('root', "create database $opt_database"); test_query('root', "use $opt_database"); # # setup test table # test_query('root', "create table $opt_table (id int primary key auto_increment, col1 int, col2 int, col3 int, col4 int)"); my @c = @cols; # column names for test database shift @c; # remove first column (id) my $cols = join ',', @c; # squash array into a string for my $i (1..N_TEST_RECORDS) { test_query('root', "insert into $opt_table ($cols) values (2, 3, 4, 5)"); } test_query('root', "grant select, insert, update, delete, create, drop, create view, show view on $opt_database.* to '$opt_user'\@'$opt_host' identified by '$opt_password'"); my $dbh_power = user_connect($opt_user, $opt_password, 0); test_query('root', "grant select, insert, update, delete on $opt_database.* to '$opt_user2'\@'$opt_host' identified by '$opt_password'"); my $dbh_crud = user_connect($opt_user2, $opt_password, 0); $roles{'power'}->[0] = $dbh_power; $roles{'crud'}->[0] = $dbh_crud; # # test views # # query array offsets use constant Q_QRY => 0; use constant Q_HI_PRIV => 1; use constant Q_LO_PRIV => 2; use constant Q_OUTPUT => 3; use constant Q_COMMENT => 4; my @t0 = ( # [ query, ignore_failure_hi_priv, ignore_failure_lo_priv, result, comment ] # [ Q_QRY, Q_HI_PRIV, Q_LO_PRIV, Q_OUTPUT, Q_COMMENT ] [ "create view $opt_table as select * from $opt_table", 1, 1, undef, "should fail - duplicate object name" ], [ "create view view1 as select * from ${opt_table}2", 1, 1, undef, "should fail - no base table found" ], [ "create view view1 as select * from $opt_table", 0, 1, undef, "" ], [ "select count(*) from view1", 0, 1, N_TEST_RECORDS, "" ], [ "create definer = current_user() sql security invoker view v1 as select 1", 0, 1, undef, "" ], [ "drop view view1", 0, 1, undef, "" ], [ "drop view v1", 0, 1, undef, "" ], ); test_driver('original create view commands', \%roles, \@t0); # # test new CREATE NO FORCE VIEW view options in 10.1.x # # CREATE FORCE VIEW should work the same as default (omitted) my @t1 = ( [ "create no force view $opt_table as select * from $opt_table", 1, 1, undef, "should fail - duplicate object name" ], [ "create no force view view1 as select * from $opt_table", 0, 1, undef, "" ], [ "create no force view view1 as select * from $opt_table", 1, 1, undef, "should fail - duplicate object name" ], [ "select count(*) from view1", 0, 1, N_TEST_RECORDS, "" ], [ "drop view view1", 0, 1, undef, "" ], ); test_driver('new CREATE NO FORCE VIEW options in 10.1.x', \%roles, \@t1); # # test new CREATE FORCE VIEW options in 10.1.x # # Similar to Oracle Enterprise, behavior of CREATE FORCE VIEW: # # - no base table needs exist at creation time # - thus no table or column access rights need exist # # - however, CREATE VIEW and SHOW VIEW should be enforced my @t2 = ( [ "create force view $opt_table as select id, col1 from $opt_table", 1, 1, undef, "should fail - duplicate object name" ], [ "create force view view1 as select id, col1 from $opt_table", 0, 1, undef, "" ], [ "create force view view1 as select id, col1 from $opt_table", 1, 1, undef, "should fail - duplicate object name" ], [ "create force view view2 as select id, col1 from ${opt_table}2", 0, 1, undef, "" ], # failed on "CREATE ALGORITHM=UNDEFINED DEFINER=`root`@`localhost` SQL SECURITY DEFINER VIEW `view1` AS select `*` AS `*` from `table1`" # note two AS symbols # [ "select count(*) from view1", 0, 1, N_TEST_RECORDS, "" ], [ "drop view view1, view2", 0, 1, undef, "" ], ); test_driver('new CREATE VIEW FORCE view options in 10.1.x', \%roles, \@t2); # test views by automatically generating thousands of permutation of CREATE VIEW gen_permutations(); # # Clean up # unlink($tmp_table); test_query('root', "revoke all privileges, grant option from '$opt_user'\@'%'"); test_query('root', "revoke all privileges, grant option from '$opt_user2'\@'%'"); test_query('root', "drop database if exists $opt_database"); print "end of test\n"; exit 0; # # do permuted tests # # Todo: # # - derived tables # - I_S tables # - views of views # - invalid syntax sub gen_permutations { # CREATE # [OR REPLACE] # [ALGORITHM = {UNDEFINED | MERGE | TEMPTABLE}] # [DEFINER = { user | CURRENT_USER }] # [SQL SECURITY { DEFINER | INVOKER }] # [[NO] FORCE] # VIEW view_name [(column_list)] # AS select_statement # [WITH [CASCADED | LOCAL] CHECK OPTION] use constant P_CREATE => 0; use constant P_ALGORITHM => 1; use constant P_DEFINER => 2; use constant P_SQL_SECURITY => 3; use constant P_FORCE => 4; use constant P_WITH => 5; # insert a blank string to make the option skippable my @p = ( [ 'create', 'create or replace'], [ '', 'undefined', 'merge', 'temptable' ], [ '', 'user', 'current_user'], [ '', 'definer', 'invoker' ], [ '', 'force', 'no force'], [ '', 'cascaded', 'local'], ); my $permutations = 0; my $sql = ''; my $quiet = 1; print "Trying create view permutations\n\n"; for my $role (sort { $b cmp $a } keys %roles) { for my $create (@{$p[P_CREATE]}) { for my $algorithm (@{$p[P_ALGORITHM]}) { for my $definer (@{$p[P_DEFINER]}) { for my $sql_security (@{$p[P_SQL_SECURITY]}) { for my $force (@{$p[P_FORCE]}) { for my $col_list (('', $cols)) { for my $columns (('*', $cols)) { for my $with (@{$p[P_WITH]}) { next if ($columns eq '*' && $force eq 'force'); # ambiguous view definition $sql = $create . ' '; $sql .= "ALGORITHM = $algorithm " if $algorithm ne ''; if ($definer ne '') { if ($definer eq 'user') { $sql .= "DEFINER = '$roles{$role}->[U_USER]'\@'$roles{$role}->[U_HOST]' "; } else { $sql .= "DEFINER = $definer "; } } $sql .= "$force " if $force ne ''; $sql .= "VIEW "; $sql .= "view1 "; $sql .= "($col_list) " if $col_list ne '' and $col_list eq $columns; # if you use $col_list, the number of columns must match $columns $sql .= "AS SELECT $columns FROM $opt_table "; $sql .= "WITH $with CHECK OPTION" if $with ne ''; print "$permutations:$role: $sql\n"; my $will_error = 0; $will_error = 1 if ($role eq 'crud') # no grants || ($algorithm eq 'temptable' and $with ne '') # always an error to CHECK a TEMPTABLE # || ($definer eq 'user' and $role ne 'root') # super priv needed for DEFINER user ; test_query('root', 'drop view if exists view1', 0, $quiet); test_query($role, $sql, $will_error, $quiet); test_query($role, 'select count(*) from view1', $will_error, $quiet); if (!$will_error && $roles{$role}->[U_PRIV_LEVEL] == HAS_VIEW_GRANTS) { if (db_cmp_count($roles{'root'}->[U_DBH], 'select count(*) from view1', N_TEST_RECORDS )) { die "error: wrong row count for '$sql'"; } } $permutations++; } } } } } } } } } print "total permutations = $permutations\n"; return 0; } sub test_driver { my ($heading, $r_roles, $r_qry) = @_; print $heading, "\n\n"; for my $role (sort { $b cmp $a } keys %$r_roles) { test_query('root', "drop view if exists view1"); for my $q (@$r_qry) { my $flag_fail = ($r_roles->{$role}->[U_PRIV_LEVEL] == HAS_VIEW_GRANTS) ? $q->[Q_HI_PRIV] : $q->[Q_LO_PRIV]; print "$role '$q->[Q_QRY]'", ($flag_fail ? ' should fail' : ' should pass'), "\n"; test_query($role, $q->[Q_QRY], $flag_fail, 0); if (defined $q->[Q_OUTPUT]) { my $ret = db_cmp_count($r_roles->{'root'}->[U_DBH], $q->[Q_QRY], ($flag_fail ? undef : $q->[Q_OUTPUT])); die if $ret; } } } print "\n\n"; } sub usage { print <<EOF; $0 Ver $version This program tests that the VIEW commands works by creating a temporary database ($opt_database) and users ($opt_user, $opt_user2). Options: --database (Default $opt_database) In which database the test tables are created. --force Don''t ask any question before starting this test. --host='host name' (Default $opt_host) Host name where the database server is located. --Information --help Print this help --root-password Password for root-user. --user (Default $opt_user) A non-existing user on which we will test view commands --password Password for non-root-user. --verbose Write all queries when we are execute them. --root-user='user name' (Default $opt_root_user) superuser for creating tables and grants EOF exit(0); } sub print_info { my $tmp; print <<EOF; This test will do view statements against the $opt_database database ! the $opt_database database and $opt_user user will be created and deleted ! EOF while (1) { print "Start test (yes/no) ? "; $tmp=<STDIN>; chomp($tmp); $tmp=lc($tmp); last if ($tmp =~ /^yes$/i); exit 1 if ($tmp =~ /^n/i); print "\n"; } } sub user_connect { my ($user, $password, $ignore_error, $db) = @_; $db = $opt_database if not defined $db or $db eq ''; print "Connecting $user\n" if ($opt_verbose); my $dbh =DBI->connect("DBI:mysql:$db:$opt_host",$user, $password, { PrintError => 0}); if (!$dbh) { if ($opt_verbose || !$ignore_error) { print "error on connect: $DBI::errstr\n"; } if (!$ignore_error) { die "The above should not have failed!"; } } elsif ($ignore_error) { die "Connect succeeded when it shouldn't have !\n"; } else { return $dbh; } } sub test_query { my ($role, $query, $ignore_error, $quiet) = @_; my ($package, $filename, $line) = caller; my $dbh = $roles{$role}->[U_DBH]; if (defined $dbh && !$dbh->ping) { $roles{$role}->[U_DBH] = user_connect($roles{$role}->[U_USER], $roles{$role}->[U_PASSWORD], $ignore_error); $dbh = $roles{$role}->[U_DBH]; } elsif (not defined $dbh) { $roles{$role}->[U_DBH] = user_connect($roles{$role}->[U_USER], $roles{$role}->[U_PASSWORD], $ignore_error); $dbh = $roles{$role}->[U_DBH]; } if (do_query($dbh, $query, $ignore_error, $quiet)) { if (!defined($ignore_error)) { print "error:$line: This query should not have failed: '$query', do SHOW CREATE VIEW VIEW_TEST.VIEW1 to troubleshoot.\n"; exit 1; } } elsif (defined($ignore_error) && $ignore_error == 1) { print "error:$line: This query should not have succeeded: '$query'\n"; exit 1; } } sub do_query { my ($my_dbh, $query, $ignore_error, $quiet) = @_; my ($sth, $row, $fatal_error); print "$query\n" if ($opt_debug || $opt_verbose); if (!($sth= $my_dbh->prepare($query))) { print "error in prepare: $DBI::errstr\n"; return 1; } if (!$sth->execute) { $fatal_error= ($DBI::errstr =~ /parse error/); if (!$ignore_error || ($opt_verbose && $ignore_error != 3) || $fatal_error) { print "error in execute: $DBI::errstr\n"; { my $cmd = "show create view $opt_database.view1"; my $out = `mysql -h $opt_host -u root -p$opt_root_password -e '$cmd'`; print "$cmd: $out\n"; print '"*** You have the SQL parser bug: AS select `*` AS `*` ***' . "\n" if $out =~ / AS SELECT .* AS /i; } } die if ($fatal_error); $sth->finish; return 1; } if (!$opt_silent and !$quiet) { my $found = 0; while (($row=$sth->fetchrow_arrayref)) { $found = 1; my $tab = ''; for my $col (@$row) { print $tab; print defined($col) ? $col : "NULL"; $tab="\t"; } print "\n"; } print "\n" if $found; } $sth->finish; return 0; } # Note: cmp_tmp_table is not currently used, but available for future use again sub cmp_tmp_table { my ($s) = @_; if (not defined $s) { if (-e $tmp_table) { return 1; } else { return 0; } } $s =~ s/\n+$//g; # remove trailing blanks open X, "<", $tmp_table or return 2; local($/)=''; my $t = <X>; close X; unlink($tmp_table) or warn "debug: cannot unlink tmp table"; $t =~ s/\n+$//g; if ($s ne $t) { return 1; } return 0; } sub db_cmp_count { my ($dbh, $q, $s) = @_; my $sth = $dbh->prepare($q); my $ret = $sth->execute() || do { return 0 if not defined $s; return 2; }; my $out = ''; if (my (@row) = $sth->fetchrow_array()) { $out = $row[0]; } $sth->finish; return 1 if $out != $s; return 0; } # The End.

1 0

[Maria-developers] mariadb native client -- how to call RPMs?
by Honza Horak 05 Sep '14

05 Sep '14

Hi, in Fedora, we're about to package mariadb-native-client, which we'd like to utilize in Fedora as much as possible since using this piece of software we can break the dependency on whole mariadb server package and thus make the base system more compact. However, since mariadb.org does not provide RPMs yet, I'm wondering what would be the best name for such RPM package. The tar ball is called mariadb_client.tar.gz but the project is called mariadb-native-client on launchpad, which I like more btw. because it differs more from mariadb-client package (which is actually a subpackage of mariadb). So, is mariadb-native-client a good choice or is there a better one? Cheers, Honza

3 3

[Maria-developers] no information about references - mysql bug 73757
by Roberto Spadim 04 Sep '14

04 Sep '14

Hi guys, there's a bug report about no precise information to solve a problem: http://bugs.mysql.com/bug.php?id=73757 test case: create table t11 (f1 integer primary key) engine innodb; alter table t11 add constraint c1 foreign key (f1) references t1(f1); /* Erro SQL (1005): Can't create table `teste`.`#sql-3f90_25c30` (errno: 150 "Foreign key constraint is incorrectly formed") */ should be better explain that reference table t1 don't exists or something like it should i report at jira, or just mysql bug database is ok? -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

2 2

[Maria-developers] Updating MySQL Client Protocol: Possible and still be backward compatible?
by Adam Scott 04 Sep '14

04 Sep '14

Is it possible to have mysql client protocol return the OS username and allow the protocol to backward compatible? Looking to see if reasonable to provide same functionality as Oracle does with select sys_context( 'userenv', 'os_user' ) from dual;

2 1

[Maria-developers] doubt about mdev 4682 - tests
by Roberto Spadim 04 Sep '14

04 Sep '14

Guys, i'm with time and was looking an old patch about query cache information schema https://mariadb.atlassian.net/browse/MDEV-4682 it add an statistic about query cache hits my doubt is... i was trying to understand how could i create tests, since the result of statistic isn't deterministic how should i execute a test case like this? columns that i have doubt: columns ... QUERY_ROWS <- deterministic SELECT_ROWS_READ <- deterministic QUERY_HITS <- deterministic QUERY_HITS_PERIOD_LOW <- non deterministic QUERY_HITS_PERIOD_HIGH <- non deterministic QUERY_HITS_PERIOD_OUTLIERS <- non deterministic QUERY_HITS_TOTAL_TIME_US <- non deterministic QUERY_HITS_MEAN_PERIOD_US <- non deterministic QUERY_HITS_MEAN_PERIOD_LOW_US <- non deterministic QUERY_HITS_MEAN_PERIOD_HIGH_US <- non deterministic QUERY_INSERT_TIME <- probably deterministic (or not?) QUERY_LAST_HIT_TIME <- probably deterministic (or not?) SELECT_EXPEND_TIME_US <- non deterministic SELECT_LOCK_TIME_US <- non deterministic ... -- Roberto Spadim

2 1

[Maria-developers] Query in size
by Kavneet Kaur 04 Sep '14

04 Sep '14

Hi all, I just when through a revision~maria-captains/mariadb-java-client/trunk : /src/test/java/org/mariadb/jdbc/BigQueryTest.java (revision 516)Big Query activated in this code contains a character array of size 20000000. This is something that can be modified. All big queries possible cannot occupy this space. This is sort of leading to more of wastage of memory. Can anyone help me understand why this size has been selected when the same could be done at a size (50-90)% lesser. Regards Kavneet Kaur

2 1

Re: [Maria-developers] Interest in contributing to MariaDB
by Djimeli Konrad 03 Sep '14

03 Sep '14

Thanks for the reply Roberto, i was beginning to get confused because the code i got from github was on c/c++. On 3 September 2014 21:13, Djimeli Konrad <djkonro35(a)gmail.com> wrote: > Thanks for the reply Roberto, i was beginning to get confused because the > code i got from github was on c/c++. > > > On 3 September 2014 20:39, Roberto Spadim <roberto(a)spadim.com.br> wrote: > >> ???? java? no guy, mariadb database source code is C/C++ check : >> >> github : https://github.com/MariaDB/server >> and issue tracker: https://mariadb.atlassian.net >> there's mariadb knowledge: https://mariadb.com/kb/en/ >> there're some book about mariadb/mysql internall and mysql internall >> documentation: http://dev.mysql.com/doc/internals/en/ >> >> java is a connect, you can check more at mariadb page: >> https://mariadb.com/products/connectors-plugins >> >> >> 2014-09-03 17:15 GMT-03:00 Kavneet Kaur <kavneetk(a)outlook.com>: >> >> Hi Djimeli, >>> >>> As far as I have studied about the code it's in java. If you wish to >>> begin it from basic of database, you can go for installation. You should go >>> through documentations also. Explore your interest and surf as much as you >>> can. >>> >>> All the best! >>> >>> > Date: Wed, 3 Sep 2014 15:35:24 +0000 >>> > From: djkonro35(a)gmail.com >>> > To: maria-developers(a)lists.launchpad.net >>> > Subject: [Maria-developers] Interest in contributing to MariaDB >>> >>> > >>> > Hello >>> > >>> > My name is Djimeli Konrad and i am primarily a c/c++ >>> > programmer with some experience in php and mysql. I am new to open >>> > source and would like to contribute to mariadb so i wish i could get >>> > some pointers on how to begin. >>> > >>> > Thanks >>> > konrad >>> > >>> > _______________________________________________ >>> > Mailing list: https://launchpad.net/~maria-developers >>> > Post to : maria-developers(a)lists.launchpad.net >>> > Unsubscribe : https://launchpad.net/~maria-developers >>> > More help : https://help.launchpad.net/ListHelp >>> >>> _______________________________________________ >>> Mailing list: https://launchpad.net/~maria-developers >>> Post to : maria-developers(a)lists.launchpad.net >>> Unsubscribe : https://launchpad.net/~maria-developers >>> More help : https://help.launchpad.net/ListHelp >>> >>> >> >> >> -- >> Roberto Spadim >> SPAEmpresarial >> Eng. Automação e Controle >> > >

1 0

[Maria-developers] Interest in contributing to MariaDB
by Djimeli Konrad 03 Sep '14

03 Sep '14

Hello My name is Djimeli Konrad and i am primarily a c/c++ programmer with some experience in php and mysql. I am new to open source and would like to contribute to mariadb so i wish i could get some pointers on how to begin. Thanks konrad

3 2

[Maria-developers] New MariaDB builds: regular trusty and special kvm-deb-sid-pbuilder-amd and (Re: MariaDB)
by Otto Kekäläinen 01 Sep '14

01 Sep '14

Hello! 2014-02-04 20:41 GMT+02:00 Daniel Bartholomew <dbart(a)mariadb.com>: > On Tue, Feb 4, 2014 at 5:39 AM, Kristian Nielsen > <knielsen(a)knielsen-hq.org> wrote: >>> And while you are at it, please also consider setting up an instance >>> 'kvm-deb-sid-pbuilder-amd64' that provides a Debian unstable >> >> Daniel, can you prepare new sets of VMs for Ubuntu trusty and Debian sid? >> Then I can try to setup Buildbot to do these builds. > > Ok, will do. I should have them up and running before the end of the week. Any updates on this?

3 22

Re: [Maria-developers] [Commits] eecac19: MDEV-6513 deprecate engine_condition_pushdown value of the @@optimizer_switch
by Oleksandr Byelkin 28 Aug '14

28 Aug '14

Hi! Everything is OK. Except one small note in two places. But it is OK to push as is. 26.08.2014 17:17, serg(a)mariadb.org пишет: > revision-id: eecac1976469d513062a0741005afc357b897871 > parent(s): f1c1c04a34f39bcd6622b9886fb089dd41e51103 > committer: Sergei Golubchik > branch nick: maria > timestamp: 2014-08-25 23:13:37 +0200 > message: > > MDEV-6513 deprecate engine_condition_pushdown value of the @@optimizer_switch > > * ignore the OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN bit > * issue a deprecation warning on 'engine_condition_pushdown=on' > * remove unused remains of the old pre-5.5 engine_condition_pushdown variable > > --- [skip] > --- a/sql/handler.h > +++ b/sql/handler.h > @@ -197,17 +197,11 @@ enum enum_alter_inplace_result { > #define HA_RECORD_MUST_BE_CLEAN_ON_WRITE (1ULL << 41) > > /* > - Table condition pushdown must be performed regardless of > - 'engine_condition_pushdown' setting. > - > - This flag is aimed at storage engines that come with "special" predicates > - that can only be evaluated inside the storage engine. > - For example, when one does > - select * from sphinx_table where query='{fulltext_query}' > - then the "query=..." condition must be always pushed down into storage > - engine. > + This storage engine supports condition pushdown > */ > -#define HA_MUST_USE_TABLE_CONDITION_PUSHDOWN (1ULL << 42) > +#define HA_CAN_TABLE_CONDITION_PUSHDOWN (1ULL << 42) > +/* old name for the same flag */ > +#define HA_MUST_USE_TABLE_CONDITION_PUSHDOWN HA_CAN_TABLE_CONDITION_PUSHDOWN > > /** > The handler supports read before write removal optimization > diff --git a/sql/mysqld.cc b/sql/mysqld.cc > index 57d2cc6..fe52a85 100644 > --- a/sql/mysqld.cc > +++ b/sql/mysqld.cc > @@ -8574,18 +8574,6 @@ mysqld_get_one_option(int optid, > } > break; > #endif /* defined(ENABLED_DEBUG_SYNC) */ > - case OPT_ENGINE_CONDITION_PUSHDOWN: > - /* > - The last of --engine-condition-pushdown and --optimizer_switch on > - command line wins (see get_options(). > - */ > - if (global_system_variables.engine_condition_pushdown) > - global_system_variables.optimizer_switch|= > - OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN; > - else > - global_system_variables.optimizer_switch&= > - ~OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN; > - break; > case OPT_LOG_ERROR: > /* > "No --log-error" == "write errors to stderr", > @@ -8964,10 +8952,6 @@ static int get_options(int *argc_ptr, char ***argv_ptr) > &extra_connection_count); > #endif > > - global_system_variables.engine_condition_pushdown= > - MY_TEST(global_system_variables.optimizer_switch & > - OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN); > - > opt_readonly= read_only; > > /* > diff --git a/sql/mysqld.h b/sql/mysqld.h > index 21c020b..7a0a75c 100644 > --- a/sql/mysqld.h > +++ b/sql/mysqld.h > @@ -554,7 +554,6 @@ enum options_mysqld > OPT_DEBUG_SYNC_TIMEOUT, > OPT_DELAY_KEY_WRITE_ALL, > OPT_DEPRECATED_OPTION, > - OPT_ENGINE_CONDITION_PUSHDOWN, > OPT_IGNORE_DB_DIRECTORY, > OPT_ISAM_LOG, > OPT_KEY_BUFFER_SIZE, > diff --git a/sql/records.cc b/sql/records.cc > index 1b230c4..cbcdb36 100644 > --- a/sql/records.cc > +++ b/sql/records.cc > @@ -287,8 +287,8 @@ bool init_read_record(READ_RECORD *info,THD *thd, TABLE *table, > thd->variables.read_buff_size); > } > /* Condition pushdown to storage engine */ > - if ((thd->variables.optimizer_switch & > - OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN) && > + if ((table->file->ha_table_flags() & > + HA_MUST_USE_TABLE_CONDITION_PUSHDOWN) && > select && select->cond && > (select->cond->used_tables() & table->map) && > !table->file->pushed_cond) > diff --git a/sql/sql_class.h b/sql/sql_class.h > index f96110e..c42a87a 100644 > --- a/sql/sql_class.h > +++ b/sql/sql_class.h > @@ -597,7 +597,6 @@ typedef struct system_variables > my_bool tx_read_only; > my_bool low_priority_updates; > my_bool query_cache_wlock_invalidate; > - my_bool engine_condition_pushdown; > my_bool keep_files_on_create; > > my_bool old_mode; > diff --git a/sql/sql_priv.h b/sql/sql_priv.h > index 5dc1918..9578c2f 100644 > --- a/sql/sql_priv.h > +++ b/sql/sql_priv.h > @@ -199,7 +199,7 @@ template <class T> bool valid_buffer_range(T jump, > #define OPTIMIZER_SWITCH_INDEX_MERGE_SORT_UNION (1ULL << 2) > #define OPTIMIZER_SWITCH_INDEX_MERGE_INTERSECT (1ULL << 3) > #define OPTIMIZER_SWITCH_INDEX_MERGE_SORT_INTERSECT (1ULL << 4) > -#define OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN (1ULL << 5) > +#define deprecated_ENGINE_CONDITION_PUSHDOWN (1ULL << 5) > #define OPTIMIZER_SWITCH_INDEX_COND_PUSHDOWN (1ULL << 6) > #define OPTIMIZER_SWITCH_DERIVED_MERGE (1ULL << 7) > #define OPTIMIZER_SWITCH_DERIVED_WITH_KEYS (1ULL << 8) > diff --git a/sql/sql_select.cc b/sql/sql_select.cc > index 05c88a5..f061b51 100644 > --- a/sql/sql_select.cc > +++ b/sql/sql_select.cc > @@ -9561,10 +9561,8 @@ make_join_select(JOIN *join,SQL_SELECT *select,COND *cond) > if (tab->table) > { > tab->table->file->pushed_cond= NULL; > - if (((thd->variables.optimizer_switch & > - OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN) || > - (tab->table->file->ha_table_flags() & > - HA_MUST_USE_TABLE_CONDITION_PUSHDOWN)) && > + if ((tab->table->file->ha_table_flags() & > + HA_MUST_USE_TABLE_CONDITION_PUSHDOWN) && It might be that it is better to use 'new' name for the flag in the lines we have changed. > !first_inner_tab) > { > COND *push_cond= > @@ -23598,10 +23596,8 @@ void JOIN_TAB::save_explain_data(Explain_table_access *eta, table_map prefix_tab > { > const COND *pushed_cond= tab->table->file->pushed_cond; > > - if (((thd->variables.optimizer_switch & > - OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN) || > - (tab->table->file->ha_table_flags() & > - HA_MUST_USE_TABLE_CONDITION_PUSHDOWN)) && > + if ((tab->table->file->ha_table_flags() & > + HA_MUST_USE_TABLE_CONDITION_PUSHDOWN) && The same. > pushed_cond) > { > eta->push_extra(ET_USING_WHERE_WITH_PUSHED_CONDITION); > diff --git a/sql/sys_vars.cc b/sql/sys_vars.cc > index 7f7c725..db764d7 100644 > --- a/sql/sys_vars.cc > +++ b/sql/sys_vars.cc > @@ -2181,13 +2181,15 @@ export const char *optimizer_switch_names[]= > "exists_to_in", > "default", NullS > }; > -/** propagates changes to @@engine_condition_pushdown */ > static bool fix_optimizer_switch(sys_var *self, THD *thd, > enum_var_type type) > { > SV *sv= (type == OPT_GLOBAL) ? &global_system_variables : &thd->variables; > - sv->engine_condition_pushdown= > - MY_TEST(sv->optimizer_switch & OPTIMIZER_SWITCH_ENGINE_CONDITION_PUSHDOWN); > + if (sv->optimizer_switch & deprecated_ENGINE_CONDITION_PUSHDOWN) > + push_warning_printf(current_thd, Sql_condition::WARN_LEVEL_WARN, > + ER_WARN_DEPRECATED_SYNTAX_NO_REPLACEMENT, > + ER(ER_WARN_DEPRECATED_SYNTAX_NO_REPLACEMENT), > + "engine_condition_pushdown=on"); > return false; > } > static Sys_var_flagset Sys_optimizer_switch( > diff --git a/storage/spider/mysql-test/spider/bg/r/spider_fixes.result b/storage/spider/mysql-test/spider/bg/r/spider_fixes.result > index 6db61fa..3033586 100644 > [skip]

2 1

Re: [Maria-developers] Learning about MariaDB, Getting Started
by Kavneet Kaur 26 Aug '14

26 Aug '14

Hi, James, I am aiming to become am interested in data security and reducing query time by increasing efficiency and computational speed. Coding and optimizing codes are also cup of my tea. How can I contribute this in MariaDB as it's the only open source database software that attracted me. Regards,Kavneet Kaur Date: Fri, 22 Aug 2014 04:44:45 -0700 From: actionmessage(a)yahoo.com Subject: Re: [Maria-developers] Learning about MariaDB, Getting Started To: kavneetk(a)outlook.com; maria-developers(a)lists.launchpad.net; ashish1294(a)gmail.com Hi Kavneet and Ashish. Depends on what you're interested in working on, but here's some suggestions.Direct link to currently open worklogs:https://mariadb.atlassian.net/browse/MDEV-6629?filter=-4Three easy MDEVs and some other areas to work on:- MDEV-6584 - not using tcp_fastopen- MDEV-6555 - sql_log_bin should be SESSION_ONLY- MDEV-5696 - Graceful MariaDB shutdown at Windows restart- test and see if fixes are needed for SSL, Galera, Cassandra, GTID, IPv6 or partition features. "Howto Add a New Command to the MySQL Server" Tutorial http://www.jebriggs.com/blog/2013/05/howto-add-a-new-command-to-the-mysql-s…, James. From: Kavneet Kaur <kavneetk(a)outlook.com> To: "maria-developers(a)lists.launchpad.net" <maria-developers(a)lists.launchpad.net> Sent: Thursday, August 21, 2014 11:49 PM Subject: [Maria-developers] Learning about MariaDB Hi! I am highly interested in subject of database. I am also interested in becoming a research developer. Worked on MS SQL. Want to try MariaDB. May I get the guidance about this product as i'm totally new to it. Is there any specific way to begin? _______________________________________________Mailing list: https://launchpad.net/~maria-developersPost to : maria-developers(a)lists.launchpad.netUnsubscribe : https://launchpad.net/~maria-developersMore help : https://help.launchpad.net/ListHelp

3 6

[Maria-developers] Please review: 10% performance improvement for INSERT INTO t1 (blob) VALUES ('long value')
by Alexander Barkov 26 Aug '14

26 Aug '14

Hi, I think I found and fixed a bottleneck in how string literals are processed. The attached patch improves INSERT performance for strings, and especially for long blob and text values. Insertion time for 10MByte blob 1024 times (10GByte of data total) reduced on my laptop from 300 seconds to 274 seconds, which gives near 10% pure time improvement. What the patch changes: Before the patch : 1. The function get_text() is sql_lex.cc allocated an unescaped copy of every string literal (unescaping backslashes and double quotes, if any). Strangely, copying happened even if there were no really any escapes in the string. 2, The syntax parser in sql_yacc.yy created Item_string using the new unescaped buffer. Furthermore, in case of a multi-byte connection character set, the constructor for Item_string made another loop on the unescaped buffer, to calculate length in characters, which is needed to set max_length properly. After the patch: Item is now created using directly the SQL fragment, without making a copy. Even for escaped values!!! Length in characters is also calculated during the very first pass in get_text(), without any additional loops in the Item constructors. Unescaping is now done in the very end, when the value is actually needed: - Either in Field::store(). This happens when the string value is used in: INSERT INTO t1 VALUES('string'); Unescaping is done directly to the Field buffer, without any intermediary temporary storage. - Or in val_str(), if the string value is used elsewhere (in SELECT list, functions, operators, etc). The unescaped value is cached, to make sure that unescaping is done only one time, even if the value is needed for multiple rows, like in: SELECT * FROM t1 WHERE a='string with backslash or quote escapes'; The patch also fixes a few bugs related to string conversion/copying (some of them were reported on JIRA earlier, and some of them I found while working on the patch). Please also find "a.c" attached. I used it to test the improvement. I used a RelWithDebInfo compiled binary for tests. Note, "perf" and "valgrind --tool=cachegrind" also demonstrate a good improvement (reduced CPU and memory usage). I can send their statistics in a separate letter, if anyone is interested. Please review the patch. Thanks.

1 0

[Maria-developers] List affecting CVEs at mariadb.com
by Otto Kekäläinen 26 Aug '14

26 Aug '14

Hello Daniel (and others), The usual changelogs[1] and relese notes[2] don't seem to contain CVE identifiers, or even a separate section about fixed security issues For the downstream security teams if would be reassuring if the CVE information would be easily available. For example if the security teams follow the CVE news and they for example know or suspect that CVE-2014-4260 affects MariaDB, it would be nice to see if it is already fixed or what version it was fixed in, so downstream security teams can organize and prioritize their patching and release work. Do you have any suggestion how to address this? Should we maybe have a separate wiki page, e.g. https://mariadb.com/kb/en/mariadb/cve/ that would have a table of CVEs and MariaDB 5.5/10.0/Galera versions where they are fixed? Or should just each release notes include a subsection "Security" with these details? Something else? Of course we need to consider timing issues, e.g. a security issue fixed in MariaDB might get publicity and a CVE only later when Oracle releases it, and in those cases old release notes need to be upgraded to include the CVE identifiers. [1] https://mariadb.com/kb/en/mariadb-10013-changelog/ [2] https://mariadb.com/kb/en/mariadb-10013-release-notes/ (To be exact, googling for 'mariadb cve' does give one hit at mariadb.com in the 5.3.12 release notes)

3 3

Re: [Maria-developers] [Commits] Rev 4271: MDEV-6601 Assertion `!thd->in_active_multi_stmt_transa ction() || thd->in_multi_stmt_transaction_mode()' failed on executing a stored procedure with commit in lp:~maria-captains/maria/5.5
by Alexey Botchkov 25 Aug '14

25 Aug '14

Hi, Sergei. I'd add SERVER_STATUS_LAST_ROW_SENT to the status_backup_mask. It's not necessary as I don't see it affecting anything, but just to make a sign that we don't want this status to go higher out of the call. Best regards. HF 25.08.2014 10:49, Sergei Golubchik wrote: > At lp:~maria-captains/maria/5.5 > > ------------------------------------------------------------ > revno: 4271 > revision-id: sergii(a)pisem.net-20140825064904-h8isjhv963kbxlzy > parent: knielsen(a)knielsen-hq.org-20140813134639-wk760plnzg5wu4x8 > fixes bug: https://mariadb.atlassian.net/browse/MDEV-6601 > committer: Sergei Golubchik <sergii(a)pisem.net> > branch nick: 5.5 > timestamp: Mon 2014-08-25 08:49:04 +0200 > message: > MDEV-6601 Assertion `!thd->in_active_multi_stmt_transa ction() || thd->in_multi_stmt_transaction_mode()' failed on executing a stored procedure with commit > > Don't restore the whole of thd->server_status after a routine invocation, > only restore SERVER_STATUS_CURSOR_EXISTS, as --ps --embedded needs. > In particular, don't restore SERVER_STATUS_IN_TRANS. > === modified file 'mysql-test/r/sp-bugs.result' > --- a/mysql-test/r/sp-bugs.result 2014-01-24 12:50:18 +0000 > +++ b/mysql-test/r/sp-bugs.result 2014-08-25 06:49:04 +0000 > @@ -268,3 +268,9 @@ END $$ > CALL test_5531(1); > DROP PROCEDURE test_5531; > DROP TABLE t1; > +create procedure sp() begin > +commit; > +end| > +start transaction; > +call sp(); > +drop procedure sp; > > === modified file 'mysql-test/t/sp-bugs.test' > --- a/mysql-test/t/sp-bugs.test 2014-01-24 12:50:18 +0000 > +++ b/mysql-test/t/sp-bugs.test 2014-08-25 06:49:04 +0000 > @@ -285,3 +285,16 @@ DELIMITER ;$$ > CALL test_5531(1); > DROP PROCEDURE test_5531; > DROP TABLE t1; > + > +# > +# MDEV-6601 Assertion `!thd->in_active_multi_stmt_transa ction() || thd->in_multi_stmt_transaction_mode()' failed on executing a stored procedure with commit > +# > +delimiter |; > +create procedure sp() begin > + commit; > +end| > +delimiter ;| > +start transaction; > +call sp(); > +drop procedure sp; > + > > === modified file 'sql/sp_head.cc' > --- a/sql/sp_head.cc 2014-04-15 13:17:47 +0000 > +++ b/sql/sp_head.cc 2014-08-25 06:49:04 +0000 > @@ -1224,6 +1224,7 @@ sp_head::execute(THD *thd, bool merge_da > Item_change_list old_change_list; > String old_packet; > uint old_server_status; > + const uint status_backup_mask= SERVER_STATUS_CURSOR_EXISTS; > Reprepare_observer *save_reprepare_observer= thd->m_reprepare_observer; > Object_creation_ctx *saved_creation_ctx; > Warning_info *saved_warning_info; > @@ -1358,7 +1359,7 @@ sp_head::execute(THD *thd, bool merge_da > It is probably safe to use same thd->convert_buff everywhere. > */ > old_packet.swap(thd->packet); > - old_server_status= thd->server_status; > + old_server_status= thd->server_status & status_backup_mask; > > /* > Switch to per-instruction arena here. We can do it since we cleanup > @@ -1488,7 +1489,7 @@ sp_head::execute(THD *thd, bool merge_da > thd->spcont->pop_all_cursors(); // To avoid memory leaks after an error > > /* Restore all saved */ > - thd->server_status= old_server_status; > + thd->server_status= (thd->server_status & ~status_backup_mask) | old_server_status; > old_packet.swap(thd->packet); > DBUG_ASSERT(thd->change_list.is_empty()); > old_change_list.move_elements_to(&thd->change_list); > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] Getting Started
by Ashish Kedia 23 Aug '14

23 Aug '14

Hi all, I am 3rd Year undergraduate student majoring in Information Technology from India. I am new to MariaDB however I wish to contribute. I am well acquainted with C++ and other tools required. Can someone help me find small tasks/bugs/projects to get started with ? Regards, Ashish Kedia, 3rd Year Undergraduate Student, Dept. of Information Technology, National Institute of Technology Karnataka, Surathkal

3 2

[Maria-developers] Closing bugs as Incomplete/Cannot reproduce requires FixVersion?
by Sergey Petrunia 23 Aug '14

23 Aug '14

Hi Serg, (writing to you because I think it was you who set Jira's rules for Fix Version field). I'm trying to close MDEV-6032 as "Incomplete" ("Cannot reproduce" would also work). This resolution is meant to say that the issue is not really resolved. And Jira's "Close Issue" dialog gives me this error: Field Fix Version/s with actual value does not match regular expression \[([0-9]+\.[0-9]+\.[0-9]+.*|N/A)\] I think it doesn't make sense. I even think that "Fix Version" must be empty for bugs that are closed with resolution other than "Fixed" or "Duplicate". Do you agree with this reasoning? If yes, is it possible to fix Jira to follow this logic? BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

2 1

[Maria-developers] Learning about MariaDB
by Kavneet Kaur 22 Aug '14

22 Aug '14

Hi! I am highly interested in subject of database. I am also interested in becoming a research developer. Worked on MS SQL. Want to try MariaDB. May I get the guidance about this product as i'm totally new to it. Is there any specific way to begin?

1 0

[Maria-developers] [GSoC] Pencils down
by sriram patil 21 Aug '14

21 Aug '14

Hi All, Following is the link to my pencils down blog post. You can go through it even if you have not followed my blog posts before. http://goo.gl/DqBgDe Thanks a lot for all your feedback and suggestions. Sriram.

3 3

[Maria-developers] [GSOC2014] UseServerPrepStmts - Connector/J
by Puneet Dewan 20 Aug '14

20 Aug '14

Hi Everyone, I have pushed the code and it can be found on the following link: https://code.launchpad.net/~team-java-connector/mariadb-java-client/mariadb… Thanks for the support. :) Please review and give suggestions. Regards Puneet.

1 0

Re: [Maria-developers] [GSoC] Optimize mysql-test-runs - Results of new strategy
by Pablo Estrada 19 Aug '14

19 Aug '14

Hello Elena and all, First, addressing the previous email: Looking at the dump, I see it can also happen that the dump contains > several records for a pair platform/bbnum. I am not sure why it happens, I > think it shouldn't, might be a bug in buildbot and/or configuration, or > environmental problems. Anyway, due to the way we store output files, they > can well override each other in this case, thus for several platform/bbnum > record you will have only one file. I suppose that's what was hard to > resolve, sorry about that. > No worries ; ). There are several cases where platform and build number are the same. The system just names the files as follows: <platform>_<build_id>-log-test_1-stdio <platform>_<build_id>-log-test_2-stdio ..... <platform>_<build_id>-log-test_5-stdio These files seem to correspond temporarily with the test runs (*test_1-stdio) belongs to the first test_run of the same plt-bnum, and so on. Unfortunately, there are some cases where there are more test_runs on the dump than files available, and this means that it's impossible to be sure which file belongs to which test_run exactly. > You should consider skipped tests, at least for now. Your logic that they > are skipped because they can't be run is generally correct; unfortunately, > MTR first produces the *full* list of tests to run, and determines whether > a test can be run or not on a later stage, when it starts running the > tests. Your tool will receive the initial test list, and I'm not sure it's > realistic to re-write MTR so that it takes into account limitations that > cause skipping tests before creating the list. > I see. Okay then, duly noted. Possibly it's better to skip a test run altogether if there is no input > list for it; it would be definitely the best if there were 5K (or whatever > slice you are currently using) of continuous test runs with input lists; if > it so happens that there are lists for some branches but not others, you > can skip the branch entirely. > This doesn't seem like a good option. Recall drops seriously, and the test_runs that have a corresponding file don't seem to have a special pattern, and tend to have long spaces between them, so the information becomes irrelevant, and seemingly, not useful. > The core module should take as parameters > - list of tests to choose from, > - size of the running set (%), > - branch/platform (if we use them in the end), > and produce a new list of tests of the size of the running set. > > The wrapper module should > - read the list of tests from the outside world (for now, from a file), > - receive branch/platform as command-line options, > - have the running set size set as an easily changeable constant or as a > configuration parameter, > > and return the list of tests -- lets say for now, in the form of <test > suite>.<test name>, blank-separated, e.g. > main.select innondb.create-index ... > >> >> I am almost done 'translating' the code into a solution that divides it in 'core' and 'wrapper'. There are a few bugs that I still haven't figured out, but I believe I can iron those out pretty soon. I will also incorporate the percentage rather than fixed running_set. Now, regarding the state of the project (and the recall numbers that I am able to achieve so far), here are some observations: - Unfortunately, I am running out of ideas to try to improve recall. I tried tuning some parameters, giving more weight to ones or others, etc. I still wasn't able to push recall beyond ~87% on the strategy that uses file correlations. For what I've seen, some failures are just extremely hard to predict. - The strategy that uses only a weighted average of the failure frequency achieves a higher recall, but for a shorter time. The recall decays quickly afterwards. I may try to add some file-correlations to this strategy, to see if the recall can be sustained for a longer term. - There is one problem that I see regarding the data and the potential real-world implementation of the program: By verifying the recall with the historical data, we run the possibility of 'expecting' overfitting... so the results regarding the errors found when comparing to the historical data, and the results that could have been obtained by a real-world implementation are potentially different. A possible way to address that issue would require modifying the buildbot to gather more data over a longer term. So having said that, I am looking for some advice in the following regards: - I will try to take a step back from the new strategy, and see how I can adapt the original strategy to prevent the recall function from declining so sharply with time. - I will also spend some time keeping a codebase that adjusts better to the model that we need for the implementation. I will upload code soon. All suggestions are welcome. - Nonetheless, I feel that more data would allow to improve the algorithm greatly. Is it possible to prepare logging into the buildbot that would allow for more precise data collection? A slower, more iterative process, working closer with the buildbot and doing more detailed data collection might deliver better results. (I understand that this would probably influence the time-scope of the project) Let me know what you think about my suggestions. Regards Pablo

2 22

[Maria-developers] [GSoC] Week Twelve!
by sriram patil 14 Aug '14

14 Aug '14

Hi All, Following is the link to my twelfth weeks blog post. http://goo.gl/Zkr86e Suggestions and/or reviews are welcome! Thanks, Sriram

3 2

[Maria-developers] Fwd: I have a replication patch, please accept it.
by Colin Charles 12 Aug '14

12 Aug '14

Xu Liang, the best place to post it is maria-developers I've done that via a FWD -- Kristian, what do you think? Begin forwarded message: > From: "Xu,Liang(OP)" <xuliang02(a)baidu.com> > Subject: I have a replication patch,please accept it. > Date: 6 August 2014 15:31:27 GMT+8 > To: "kb-admins(a)askmonty.org" <kb-admins(a)askmonty.org> > > http://bugs.mysql.com/bug.php?id=73479 > > > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > Baidu DBA 徐良 > 分机：86912826 > 手机：18689220441 > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Colin Charles, Chief Evangelist, SkySQL - The MariaDB Company blog: http://bytebot.net/blog/| t: +6-012-204-3201 | Skype: colincharles

1 0

[Maria-developers] MariaDB 10.0.13 Release Prep
by Daniel Bartholomew 11 Aug '14

11 Aug '14

I've begun the prep work for the MariaDB 10.0.13 release. Draft changelog and release notes are here: - https://mariadb.com/kb/en/mariadb-10013-release-notes/ - https://mariadb.com/kb/en/mariadb-10013-changelog/ A big thanks to those that have already contributed content! As always, please let me know ASAP if anything else needs to be added/changed/etc..., or jump in and do it yourself. Thanks! -- Daniel Bartholomew, MariaDB Release Manager MariaDB | http://mariadb.com

1 0

[Maria-developers] innodb-buffer-pool-instances confusion
by Daniel Bartholomew 07 Aug '14

07 Aug '14

I was chatting with some folks on IRC and someone pointed this out to me. On the "Upgrading From MariaDB 5.5 to MariaDB 10.0" page, it says that the new default value for innodb-buffer-pool-instances is '0': https://mariadb.com/kb/en/upgrading-from-mariadb-55-to-mariadb-100/ And when I run the following, it says the default is indeed '0': mysqld --no-defaults --verbose --help | grep innodb-buffer-pool-instances ... innodb-buffer-pool-instances 0 But when I'm in the client, I get the following: MariaDB [(none)]> select @@innodb_buffer_pool_instances; +--------------------------------+ | @@innodb_buffer_pool_instances | +--------------------------------+ | 8 | +--------------------------------+ So when innodb-buffer-pool-instances is set to '0', does that mean the actual value is dynamic or calculated in some way based on my other settings? Just trying to understand what's going on, and once I do, use it to update the XtraDB/InnoDB buffer-pool page (which doesn't appear to have been updated for 10.0). https://mariadb.com/kb/en/xtradbinnodb-buffer-pool/ Thanks! -- Daniel Bartholomew, MariaDB Release Manager MariaDB | http://mariadb.com

4 3

[Maria-developers] CONNECT system variables
by Ian Gilfillan 07 Aug '14

07 Aug '14

Hi I am trying to document the CONNECT system variables at https://mariadb.com/kb/en/mariadb/documentation/storage-engines/connect/con… but I can't seem to find much documentation in the code. I managed to put something together from Olivier's notes in JIRA for some of them, but can anyone help fill in the gaps and send me details on the other variables? thanks, ian

1 0

[Maria-developers] a question about group commit
by nanyi607rao 07 Aug '14

07 Aug '14

Hi, as we know there are 3 steps in XA transaction committing 1, prepare step 2, write binary log 3, commit step in engines all these steps need a fsync(). Group commit strategy can make a group of transactions durable with one fsync() at step 2 and step 3, which can lead to dramatic performance enchance. But in step 1, each transaction still do its own fsync(). so why not make several transactions durable whith one fsync() in prepare step just like step 2 and 3, which I think can improve performanc further more ? Thanks 2014-08-04 nanyi607rao

2 3

[Maria-developers] [GSoC] Week Eleven!
by sriram patil 05 Aug '14

05 Aug '14

Hi All, Following is the link to the blog post for my eleventh week into GSoC http://goo.gl/TN3U0K Suggestions and/or reviews are welcome. Thanks, Sriram

3 2

[Maria-developers] MariaDB 5.5.39 Release Prep
by Daniel Bartholomew 05 Aug '14

05 Aug '14

All, I've begun prep work on the MariaDB 5.5.39 release. Draft release notes and changelog are here: - https://mariadb.com/kb/en/mariadb-5539-changelog/ - https://mariadb.com/kb/en/mariadb-5539-release-notes/ Please add any necessary or notable items to the release notes ASAP. Current plan is to activate/announce the release on 5 Aug. Thanks! -- Daniel Bartholomew, MariaDB Release Manager MariaDB | http://mariadb.com

1 0

[Maria-developers] Connector Coding till 5th Aug
by Puneet Dewan 05 Aug '14

05 Aug '14

Hi Everyone, Last week, 1.I was working on the Binary Protcol Resultset which is still going on. 2.Added code for Batch Statements in MySQlServerSidePreparedStatement. 3.Added utility methods in ParameterWriter.java for sending different parameters in Binary Protocol format. 4.Code review with Massimo This week, 1.Complete the Binary protocol resultset. 2.Test cases for batch statements and allowmultiqueries. 3.Code review with Massimo. Regards Puneet.

1 0

[Maria-developers] [GSoC] Introduction Mail
by Anshu Avinash 04 Aug '14

04 Aug '14

Hi all, I have been accepted for the project "Self-Tuning Optimizer". I will be doing this project under the mentorship of Sergei Golubchik. There is already a task created which will track the progress for this project: https://mariadb.atlassian.net/browse/MDEV-350 . I'll start with getting familiar with the code base, reading some literature on cost estimation in optimizer and interacting with community. About me: I am currently a 4th year student pursuing B.Tech - M.Tech dual degree in Computer Science and Engineering from Indian Institute of Technology (IIT) – Kanpur, India. This would be my first GSoC project. I had done an internship at Facebook last year. I'll also be maintaining a blog at http://igniting.in . I'm really excited to be a part of mariadb and I hope to make a significant contribution to it. Regards Anshu Avinash

5 48

[Maria-developers] Review request MDEV-6274 Collation usage statistics (for feedback plugin)
by Alexander Barkov 30 Jul '14

30 Jul '14

Hello Sergei, Please review a patch for MDEV-6274. Thanks.

2 3

Re: [Maria-developers] [Commits] Rev 4227: MDEV-6441: memory leak in file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6441/
by Igor Babaev 29 Jul '14

29 Jul '14

Sanja, Correct the comment for the commit and push. The only question I have concerns another comment: +# MDEV-5515: sub-bug nets of 3rd execution crash ^ Is this an intended word? Regards, Igor. On 07/28/2014 05:36 AM, sanja(a)askmonty.org wrote: > At file:///home/bell/maria/bzr/work-maria-5.5-MDEV-6441/ > > ------------------------------------------------------------ > revno: 4227 > revision-id: sanja(a)askmonty.org-20140728123541-3dh3f20p41iyrefm > parent: sanja(a)askmonty.org-20140715093734-42nrug10i0xw8hvb > committer: sanja(a)askmonty.org > branch nick: work-maria-5.5-MDEV-6441 > timestamp: Mon 2014-07-28 15:35:41 +0300 > message: > MDEV-6441: memory leak > > Whole mysql_derived_prepare() was executed on statement memory. Now it is executed on runtime meory and all bugs induced bu this fixed. mysql_derived_prepare() was executed on the statement memory. Now it is executed on the runtime memory. All bugs induced by this were fixed. > > > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

1 0

[Maria-developers] Performance: question about strings created in sql_lex.cc
by Alexander Barkov 29 Jul '14

29 Jul '14

Hello, When parsing an SQL query, the function get_text() in sql_lex.cc always makes a copy of the string, allocating new memory using lip->m_thd->alloc() and copying the original string from the SQL query fragment to this new memory. Note, this happens even if the string does not have any escaped characters! I'm seeing some performance bottleneck here. It would be nice to avoid copying. Why could not we point a non-escaped string to the original SQL query fragment where the string is actually written? Are there any situations when the original SQL query is not accessible any more, but Item_string created at parse time is still in use? Are there any other things that are created in sql_yacc.yy using the TEXT_STRING token, which could exist longer than the original SQL query? Thanks.

1 0

[Maria-developers] merge 10.0-connect -> 10.0
by Alexander Barkov 29 Jul '14

29 Jul '14

Hello Olivier, I tried to merge 10.0-connect into the main 10.0, but two mtr tests failed in the merged tree. Note, they don't fail in 10.0-connect, so I guess something have changed in the main tree which makes your latest changes in 10.0-connect incompatible. Please find the log attached. I have no idea how to fix the failures. Can I push the merge into 10.0-connect, so you can fix the problems? Also, I noticed that you changed the sources outside of the "/storage/connect" directory: > --- sql/sql_partition.cc 2014-03-26 21:25:38 +0000 > +++ sql/sql_partition.cc 2014-07-29 04:44:43 +0000 > @@ -3195,13 +3195,26 @@ uint32 get_partition_id_cols_list_for_en > (list_index - 1)*num_columns, > nparts, left_endpoint, > include_endpoint))); > - > +#if 0 > if (!left_endpoint) > { > /* Set the end after this list tuple if not already after the last. */ > if (list_index < part_info->num_parts) > list_index++; > } > +#else > + if (!left_endpoint && list_index < part_info->num_list_values) > + { > + /* > + Set the end after this list tuple if it is not already after the last > + and it matches. > + */ > + int cmp = cmp_rec_and_tuple_prune(list_col_array + list_index*num_columns, > + nparts, left_endpoint, include_endpoint); > + if (cmp >= 0) > + list_index++; > + } > +#endif > > DBUG_RETURN(list_index); > } Can you please comment why do you need this change? Did you discuss this change with anyone? Thanks.

1 0

[Maria-developers] Connector Coding till 28th July
by Puneet Dewan 28 Jul '14

28 Jul '14

Hi Everyone, Last week, 1.Migrated the code from MySQLPreparedStatement to MySQLServerSidePreparedStatement. 2.Removed duplicate code in classes like IntParameter, DoubleParameter, LongParameter and used methods of class ParameterWriter(provides utility methods) in the above classes. 3.Did the analysis of TextProtocol Resultset has been implemented in connector code and find a way on how to write for Binary Protocol Resultset. 4.Fixed the new_params_bound_flag issue. 5.Code review with Massimo. This week, 1.Code review with Massimo 2.Complete the coding for Binary Protocol Resultset. 3.Write code for allowMultiQueries, Batchstatements. Regards Puneet.

1 0

[Maria-developers] [GSoC] Week Nine and Ten!
by sriram patil 28 Jul '14

28 Jul '14

Hi All, Following is the blog link about my ninth and tenth week into GSoC. http://goo.gl/N5Le32 Suggestions and/or reviews are welcome! Thanks, Sriram

1 0

[Maria-developers] MDEV-4511 Assertion `scale <= precision' fails on GROUP BY TIMEDIFF with incorrect types
by Alexander Barkov 25 Jul '14

25 Jul '14

Hello Sergei, Please review a new version of the patch for MDEV-4511, it also fixes more crashing problems found, as well as non-crashing problems reported in MDEV-6302 I removed find_num_type() and fix_num_length_and_dec() which were hard to track, and for some Items these methods were erroneously called twice. fix_num_length_and_dec() now only exists as a non-virtual method in Item_sum and Item_udf_func. The patch is for 5.3, but it is still valid for 10.0, as we still have temporal functions with NOT_FIXED_DEC in 10.0. Thanks.

2 3

[Maria-developers] Any plans about starting to use /var/lib/mariadb for data instead of /var/lib/mysql?
by Otto Kekäläinen 24 Jul '14

24 Jul '14

Hello! Does the MariaDB project have any plans about switching the data dir to /var/lib/mariadb in some release? In Debian there has been discussion about separating the binary-incompatible MySQL forks into separate /var/lib/<fork> to ease maintenance and secure data so that purging some packages will not accidentally delete data that another package uses. The thread is named "Discussion on handling 5.6 related matters" and available at http://lists.alioth.debian.org/pipermail/pkg-mysql-maint/2014-July/thread.h… I'd appreciate any guidelines here on what the upstream plans are. The Debian MySQL/MariaDB packaging team willl have an IRC meeting tomorrow at 16:00 UTC and I need to express my opinion on the mattes and what kind of changes I'm willing to do to the MariaDB package. Obviously I'd like to follow upstream as much as possible to keep the delta between Debian and upstream small and thus the maintenance burden reasonable. -- Check out our blog at http://seravo.fi/blog and follow @ottokekalainen

3 2

[Maria-developers] please review mdev-5745 analyze MySQL fix for bug#12368495
by Alexander Barkov 23 Jul '14

23 Jul '14

Hi Serg, please review a patch for mdev-5745. Thanks.

1 0

Re: [Maria-developers] [Commits] 00ed367: MDEV-4262 - P_S discovery
by Sergei Golubchik 22 Jul '14

22 Jul '14

Hi, Sergey! On Jul 07, svoj(a)mariadb.org wrote: > revision-id: 00ed36700addfbce91940223cce4ccd6ee5823a1 > parent(s): f37f19a6c790100cc2ae5f483ba4db34a517f0ae > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2014-07-07 14:32:13 +0400 > message: > > MDEV-4262 - P_S discovery > > Discover P_S tables automatically. > > Most of this patch is code clean-up: > - removed tests and code responsible for P_S tables correctness verification > - always return error from ha_perfschema::create() > - install/upgrade scripts won't create P_S tables anymore Great work, thanks! The patch is pretty much "ok to push", but, first I want to understand why you deleted table_file_summary.cc file. See other minor comments below. > diff --git a/mysql-test/suite/perfschema/include/upgrade_check.inc b/mysql-test/suite/perfschema/include/upgrade_check.inc > index 52d4cfd..1532de8 100644 > --- a/mysql-test/suite/perfschema/include/upgrade_check.inc > +++ b/mysql-test/suite/perfschema/include/upgrade_check.inc > @@ -3,7 +3,6 @@ > # > > --source include/count_sessions.inc > ---error 1 > --exec $MYSQL_UPGRADE --skip-verbose --force > $out_file 2> $err_file > --source include/wait_until_count_sessions.inc The rest of this upgrade_check.inc file does (according to the comment) "Verify that mysql_upgrade complained about the performance_schema" perhaps you should change the comment to say "does not complain" ? > > diff --git a/scripts/mysql_performance_tables.sql b/scripts/mysql_performance_tables.sql > index d495578..3061603 100644 > --- a/scripts/mysql_performance_tables.sql > +++ b/scripts/mysql_performance_tables.sql > @@ -53,1427 +53,3 @@ SET @str = IF(@broken_pfs = 0, @cmd, 'SET @dummy = 0'); > PREPARE stmt FROM @str; > EXECUTE stmt; > DROP PREPARE stmt; Why did you keep the above? I'd think that a simple DROP DATABASE IF EXISTS performance_schema should be enough to replace the whole file. And it could be moved back to mysql_system_tables.sql, so this file could be completely removed. > - > -# > -# From this point, only create the performance schema tables > -# if the server is built with performance schema > -# > - > -set @have_pfs= (select count(engine) from information_schema.engines where engine='PERFORMANCE_SCHEMA' and support != 'NO'); > - > -# > -# TABLE COND_INSTANCES > -# ... > diff --git a/storage/perfschema/pfs_engine_table.cc b/storage/perfschema/pfs_engine_table.cc > index 958a2bd..6f0a363 100644 > --- a/storage/perfschema/pfs_engine_table.cc > +++ b/storage/perfschema/pfs_engine_table.cc > @@ -1440,5 +1319,17 @@ end: > DBUG_RETURN(false); > } > > +int pfs_discover_table_names(handlerton *hton, LEX_STRING *db, > + MY_DIR *dir, hm, no "unused argument" compiler warnings? okay... > + handlerton::discovered_list *result) > +{ > + if (compare_table_names(db->str, PERFORMANCE_SCHEMA_str.str)) > + return 0; > + for (size_t i= 0; i < array_elements(all_shares) - 1; i++) > + result->add_table(all_shares[i]->m_name.str, > + all_shares[i]->m_name.length); > + return 0; > +} > + > /** @} */ > > diff --git a/storage/perfschema/table_file_summary.cc b/storage/perfschema/table_file_summary.cc > deleted file mode 100644 > index 104fa0f..0000000 > --- a/storage/perfschema/table_file_summary.cc > +++ /dev/null > @@ -1,372 +0,0 @@ Why did you remove this file? > -/* Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. > - > - This program is free software; you can redistribute it and/or modify > - it under the terms of the GNU General Public License as published by > - the Free Software Foundation; version 2 of the License. Regards, Sergei

2 4

Re: [Maria-developers] MDEV-4904 - Add query cache hit information to Percona Response time distribution plugin (MDEV-4568)
by Sergei Golubchik 22 Jul '14

22 Jul '14

Hi, svoj! On Jun 26, svoj(a)mariadb.org wrote: > revision-id: f37f19a6c790100cc2ae5f483ba4db34a517f0ae > parent(s): 4a3f0684195703996b23c51af79cf946b85b8053 > committer: Sergey Vojtovich > branch nick: 10.1 > timestamp: 2014-06-26 16:37:46 +0400 > message: > > MDEV-4904 - Add query cache hit information to Percona Response time distribution plugin (MDEV-4568) > > Added two new fields to INFORMATION_SCHEMA.QUERY_RESPONSE_TIME: > QC_HITS_COUNT - query cache hits count > QC_HITS_TOTAL - cumulative response time for query cache hits. This is somewhat questionable feature. Queries served from qc don't take much time, which means that @@query_response_time_range_base should be different for them. Otherwise all qc hits will be in the first few slots. But a different base means a different table (QUERY_CACHE_RESPONSE_TIME ?) and a different set of configuration variables. On the other hand, this current implementation is much simpler. So I think we can accept this patch. But I don't know how useful it'll be. Alternatively, you can do a separate I_S table for QC. As you like. > diff --git a/plugin/query_response_time/mysql-test/query_response_time/query_response_time-stored.result b/plugin/query_response_time/mysql-test/query_response_time/query_response_time-stored.result > index bec7007..e487fc5 100644 > --- a/plugin/query_response_time/mysql-test/query_response_time/query_response_time-stored.result > +++ b/plugin/query_response_time/mysql-test/query_response_time/query_response_time-stored.result > @@ -40,51 +40,51 @@ SHOW GLOBAL VARIABLES where Variable_name like 'QUERY_RESPONSE_TIME_RANGE_BASE'; > Variable_name Value > query_response_time_range_base 2 > SELECT * FROM INFORMATION_SCHEMA.QUERY_RESPONSE_TIME; I'd rather replaced * with TIME,COUNT,TOTAL - less changes and more future proof. > -TIME COUNT TOTAL > - 0.000001 45 0.000000 > +TIME COUNT TOTAL QC_HITS_COUNT QC_HITS_TOTAL > + 0.000001 45 0.000000 0 0.000000 ... Regards, Sergei

1 0

[Maria-developers] Why does name resolution stop in crash stack trace?
by Sergey Petrunia 21 Jul '14

21 Jul '14

Hi, Look at today's crash report in maria-discuss@. A question not related to the cause of that crash: why do stack traces that go inside the storage engine cannot be resolved? Is something wrong with our build process? On Fri, Jun 13, 2014 at 08:15:39PM -0700, grantksupport(a)operamail.com wrote: > ==> /var/log/mariadb/mariadb-err.log <== > 140613 20:07:24 [ERROR] mysqld got signal 11 ; ... > Server version: 10.0.11-MariaDB-log > key_buffer_size=268435456 > read_buffer_size=1048576 > max_used_connections=1 > max_threads=66 > thread_count=1 > It is possible that mysqld could use up to > key_buffer_size + (read_buffer_size + > sort_buffer_size)*max_threads = 601410 K bytes of > memory > Hope that's ok; if not, decrease some variables in the > equation. > > Thread pointer: 0x0x7fcf132fa008 > Attempting backtrace. You can use the following > information to find out > where mysqld died. If you see no messages after this, > something went > terribly wrong... > stack_bottom = 0x7fcf4a6e4e78 thread_stack 0x100000 > /usr/local/mariadb/bin/mysqld(my_print_stacktrace+0x2e)[0xacb60e] > /usr/local/mariadb/bin/mysqld(handle_fatal_signal+0x390)[0x6f5a10] > /lib64/libpthread.so.0(+0xf9f0)[0x7fcf4ff2a9f0] > /usr/local/mariadb/bin/mysqld[0x915cba] > /usr/local/mariadb/bin/mysqld[0x909e27] > /usr/local/mariadb/bin/mysqld[0x90a35b] > /usr/local/mariadb/bin/mysqld[0x90c759] > /usr/local/mariadb/bin/mysqld[0x9169d5] > /usr/local/mariadb/bin/mysqld[0x8aaf2b] > /usr/local/mariadb/bin/mysqld[0x8ab3cf] > /usr/local/mariadb/bin/mysqld[0x8ab95a] > /usr/local/mariadb/bin/mysqld[0x895e80] > /usr/local/mariadb/bin/mysqld[0x89643f] > /usr/local/mariadb/bin/mysqld[0x9657da] > /usr/local/mariadb/bin/mysqld[0x965a16] > /usr/local/mariadb/bin/mysqld[0x96a51d] > /usr/local/mariadb/bin/mysqld[0x96ac28] > /usr/local/mariadb/bin/mysqld[0x82de54] > /usr/local/mariadb/bin/mysqld(_ZN7handler9ha_createEPKcP5TABLEP14HA_CREATE_INFO+0x70)[0x6fcbf0] > /usr/local/mariadb/bin/mysqld(_Z15ha_create_tableP3THDPKcS2_S2_P14HA_CREATE_INFOP34st_mysql_const_unsigned_lex_string+0x220)[0x6fd560] > /usr/local/mariadb/bin/mysqld(_Z16rea_create_tableP3THDP34st_mysql_const_unsigned_lex_stringPKcS4_S4_P14HA_CREATE_INFOP7handlerb+0xbb)[0x659d1b] > /usr/local/mariadb/bin/mysqld[0x62e50d] > /usr/local/mariadb/bin/mysqld(_Z26mysql_create_table_no_lockP3THDPKcS2_P14HA_CREATE_INFOP10Alter_infoPbi+0xd3)[0x62ee13] > /usr/local/mariadb/bin/mysqld(_Z18mysql_create_tableP3THDP10TABLE_LISTP14HA_CREATE_INFOP10Alter_info+0xb6)[0x62ef76] > /usr/local/mariadb/bin/mysqld(_Z21mysql_execute_commandP3THD+0x672d)[0x5be04d] > /usr/local/mariadb/bin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x1d7)[0x5bf4f7] > /usr/local/mariadb/bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x15d7)[0x5c1057] > /usr/local/mariadb/bin/mysqld(_Z24do_handle_one_connectionP3THD+0x20c)[0x66f0ec] > /usr/local/mariadb/bin/mysqld(handle_one_connection+0x38)[0x66f138] > /usr/local/mariadb/bin/mysqld[0x9d5184] > /lib64/libpthread.so.0(+0x80db)[0x7fcf4ff230db] > /lib64/libc.so.6(clone+0x6d)[0x7fcf4ebd390d] > > Trying to get some variables. > Some pointers may be invalid and cause the dump to > abort. > Query (0x7fcf1143d020): CREATE TABLE menu_router ( BR Sergei -- Sergei Petrunia, Software Developer MariaDB | Skype: sergefp | Blog: http://s.petrunia.net/blog

2 1

[Maria-developers] [GSoC] Week Nine!
by sriram patil 21 Jul '14

21 Jul '14

Hi All, There will be a little delay in the ninth week blog post. I need to get the code reviewed first to make the final changes. You can check the commit here <https://github.com/srirampatil/server/commit/3611bfc644714b797acff8b9884e7d…> though. I have also fixed MDEV-6409 <https://mariadb.atlassian.net/browse/MDEV-6409> and it is under review too. Regards, Sriram

1 0

[Maria-developers] Java Connector Coding till 20th July
by Puneet Dewan 20 Jul '14

20 Jul '14

Hi Everyone, Sorry,I did not update in the last week. It is because of my convocation and was meeting my parents after one year in Singapore, so enjoyed a small vacation of 5 days with them. I had code review and discussion with Massimo in that week. But I thought it is better to update next week. Things done: 1.I had initially written code in MySQLPreparedStatement.java but the code was getting ugly so as per recommnedation from Vladislav,I also discussed with my mentor.So we thought of using MySQLServerSidePreparedStatement as this will make code more cleaner. Now prepareStatement method in MySQLConnection will become the factory method to provide either MySQLPreparedStatement or MySQLServerSidePreparedStatement instance. I have migrated my code to MySQLServerSidePreparedStatement. 2.Test case for determining the type of instance, MySQLPreparedStatement or MySQLServerSidePreparedStatement. Next plan: 1.Write executeQuery and executeUpdate methods 2.For executeQuery, need to decide how I can use BinaryProtocolResultset to fetch results from a select query. 3.Write methods for batchstatements and multiqueries. Regards Puneet.

1 0

[Maria-developers] [GSoC] Optimize mysql-test-runs - Results of new strategy
by Pablo Estrada 17 Jul '14

17 Jul '14

Hello everyone, I took the last couple of days working on a new strategy to calculate the relevance of a test. The results are not sufficient by themselves, but I believe they point to an interesting direction. This strategy uses that rate of co-occurrence of events to estimate the relevance of a test, and the events that it uses are the following: - File editions since last run - Test failure in last run The strategy has also two stages: 1. Training stage 2. Executing stage In the training stage, it goes through the available data, and does the following: - If test A failed: - It counts and stores all the files that were edited since the last test_run (the last test_run depends on BRANCH, PLATFORM, and other factors) - If test A failed also in the previous test run, it also counts that In the executing stage, the training algorithm is still applied, but the decision of whether a test runs is based on its relevance, the relevance is calculated as the sum of the following: - The percentage of times a test has failed in two subsequent test_runs, multiplied by whether the test failed in the previous run (if the test didn't fail in the previous run, this quantity is 0) - For each file that was edited since the last test_run, the percentage of times that the test has failed after this file was edited (The explanation is a bit clumsy, I can clear it up if you wish so) The results have not been too bad, nor too good. With a running set of 200 tests, a training phase of 3000 test runs, and an executing stage of 2000 test runs, I have achieved recall of 0.50. It's not too great, nor too bad. Nonetheless, while running tests, I found something interesting: - I removed the first factor of the relevance. I decided to not care about whether a test failed in the previous test run. I was only using the file-change factor. Naturally, the recall decreased, from 0.50 to 0.39 (the decrease was not too big)... and the distribution of failed tests in the priority queue had a good skew towards the front of the queue (so it seems that the files help somewhat, to indicate the likelihood of a failure). I attached this chart. An interesting problem that I encountered was that about 50% of the test_runs don't have any file changes nor test failures, and so the relevance of all tests is zero. Here is where the original strategy (a weighted average of failures) could be useful, so that even if we don't have any information to guess which tests to run, we just go ahead and run the ones that have failed the most, recently. I will work on mixing up both strategies a bit in the next few days, and see what comes of that. By the way, I pushed the code to github. The code is completely different, so may be better to wait until I have new results soon. Regards! Pablo

2 11

[Maria-developers] Java Connector Coding till 5th June
by Puneet Dewan 17 Jul '14

17 Jul '14

Hi Everyone, The work done till 5th June: 1.I started studying the connector immediately after my semester exams got over(5th May). 2. Understood the relationship between different classes. 3.Read the MySql documentation for text protocol and map the commands from documentation onto code.This thing gave a clear idea about how things are moving from client to server. 4.Setup the code on Ubuntu using eclipse and tested against database. Setup the Team branch in launchpad. 5.Debug the test classes to see how different classes are interlinked and understood that MySQLProtocol.java is the heart of our JDBC driver. 6.Then I started reading Client Server Protocol documentation i.e ServerSide prepared statement and try to see how and where I have to code it. 7.Then I saw that there is a class MySqlServerSidePreparedStatement.java, Initially I coded the prepareStatement() ,close(),and execute() in that class seperately. But after (Code review) discussion with Massimo, we donot need to use that class. Instead use MySqlPreparedStatement.java to do the same work.So made changes accordingly. 8.I had pushed some of the code which I have written to launchpad and discussed with Massimo last week. 9.Last week I was struck with Wireshark on windows and ubuntu both. I was unable to see the packets going to database and seeing the response. 10.On Monday the problem was resolved as I was using the wrong interface.I should have used loopback.io(127.0.0.1) instead on ubuntu(thanks to Georg). Unfortunately the thing does not work on windows.Tried a lot many times. 11.Now I am working on the execute method of client server protocol as I am getting exception while using that method.Wireshark is playing a great role in how the packets are flowing. >From now I will do weekly blogging and keep everyone updated. Thanks & Regards Puneet.

2 9

[Maria-developers] Help needed TokuDB build errors in Debian
by Otto Kekäläinen 16 Jul '14

16 Jul '14

Hello Rich! For some strange reason, re-building 5.5.37 in Debian unstable stopped working. Any ideas what might be going on here?: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=753222 Recently a build of 5.5.38 generated a similar error: http://paste.ubuntu.com/7747594/ Are these maybe related? I have no idea how to fix these apart from disabling TokuDB in Debian when uploading 5.5.38. -- Check out our blog at http://seravo.fi/blog and follow @ottokekalainen

3 5

[Maria-developers] [GSoC] Week Eight!
by sriram patil 14 Jul '14

14 Jul '14

Hi all, Following is the blog link about my eighth week into GSoC. http://goo.gl/h2mnbv Suggestions and/or reviews are welcome! Thanks, Sriram

1 0

Re: [Maria-developers] [Commits] Rev 4876: Post-fixes after UNIQUE index support was enabled without uniquenessenforcement. in file:///home/psergey/chroot/saucy-x64/home/psergey/dev2/mysql-5.6-rocksdb-r2/
by Roberto Spadim 11 Jul '14

11 Jul '14

=] nice rocksdb, any devel code to test it? i have some key-value databases to test 2014-07-11 13:21 GMT-03:00 Sergey Petrunya <psergey(a)askmonty.org>: > At > file:///home/psergey/chroot/saucy-x64/home/psergey/dev2/mysql-5.6-rocksdb-r2/ > > ------------------------------------------------------------ > revno: 4876 > revision-id: psergey(a)askmonty.org-20140711162114-051drr4tocqpbx30 > parent: psergey(a)askmonty.org-20140711161708-b50tdmlr60djrror > committer: Sergey Petrunya <psergey(a)askmonty.org> > branch nick: mysql-5.6-rocksdb-r2 > timestamp: Fri 2014-07-11 20:21:14 +0400 > message: > Post-fixes after UNIQUE index support was enabled without > uniquenessenforcement. > - Fix compile warning > - Fix the rocksdb.test/result > === modified file 'mysql-test/r/rocksdb.result' > --- a/mysql-test/r/rocksdb.result 2014-07-03 20:58:25 +0000 > +++ b/mysql-test/r/rocksdb.result 2014-07-11 16:21:14 +0000 > @@ -686,8 +686,8 @@ DROP TABLE t33; > # > # MDEV-4081: RocksDB throws error 122 on an attempt to create a table > with unique index > # > +# Unique indexes can be created, but uniqueness won't be enforced > create table t33 (pk int primary key, u int, unique index(u)) > engine=RocksDB; > -ERROR HY000: Got error 187 'Unique indexes are not supported' from ROCKSDB > # > # MDEV-4077: RocksDB: Wrong result (duplicate row) on select with range > # > > === modified file 'mysql-test/t/rocksdb.test' > --- a/mysql-test/t/rocksdb.test 2014-07-03 20:58:25 +0000 > +++ b/mysql-test/t/rocksdb.test 2014-07-11 16:21:14 +0000 > @@ -610,7 +610,8 @@ DROP TABLE t33; > --echo # > --echo # MDEV-4081: RocksDB throws error 122 on an attempt to create a > table with unique index > --echo # > ---error ER_GET_ERRMSG > +#--error ER_GET_ERRMSG > +--echo # Unique indexes can be created, but uniqueness won't be enforced > create table t33 (pk int primary key, u int, unique index(u)) > engine=RocksDB; > > > > === modified file 'storage/rocksdb/ha_rocksdb.cc' > --- a/storage/rocksdb/ha_rocksdb.cc 2014-07-09 23:46:24 +0000 > +++ b/storage/rocksdb/ha_rocksdb.cc 2014-07-11 16:21:14 +0000 > @@ -1051,7 +1051,7 @@ int ha_rocksdb::create(const char *name, > /* > We only support auto_increment at start of the PRIMARY KEY. > */ > - Field *field; > + //Field *field; > //if ((field= table_arg->next_number_field)) > /* TODO mdcallag: disable this for now to let UNIQUE indexes kind of > work > if ((field= table_arg->found_next_number_field)) > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits > -- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

1 0

Re: [Maria-developers] apt-get upgrade on buildbot
by Kristian Nielsen 11 Jul '14

11 Jul '14

Otto Kekäläinen <otto(a)seravo.fi> writes: > Hello Kristian! > > Could you do a small favour and run 'apt-get upgrade' and then trigger > manually a rebuild in the virtual machine Debian unstable the buildbot > uses at http://buildbot.askmonty.org/buildbot/builders/debpkg-sid > > Thanks! Hm, I tried this, but I got the below errors. Any ideas what I did wrong? - Kristian. buildbot@debian-sid-amd64:~$ sudo apt-get update Hit http://ftp.us.debian.org sid InRelease Hit http://ftp.us.debian.org sid/main Sources/DiffIndex Hit http://ftp.us.debian.org sid/main amd64 Packages/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-en/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-cs/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-ru/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-nl/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-eo/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-ca/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-pl/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-fi/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-hr/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-fr/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-eu/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-sr/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-hu/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-da/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-ro/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-es/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-pt/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-sv/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-sk/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-ko/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-vi/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-uk/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-km/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-ja/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-de/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-it/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-nb/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-zh/DiffIndex Hit http://ftp.us.debian.org sid/main Translation-id/DiffIndex Get:1 http://ftp.us.debian.org sid/main Translation-ru [423 kB] Get:2 http://ftp.us.debian.org sid/main Translation-pl [575 kB] 100% [2 Translation-pl bzip2 0 B] 66.3 kB/s 0sFalling back to normal index file acquire Ign http://ftp.us.debian.org sid/main 2014-07-09-0854.18.pdiff Get:3 http://ftp.us.debian.org sid/main Translation-fr [731 kB] 100% [3 Translation-fr bzip2 0 B] 90.5 kB/s 0sFalling back to normal index file acquire Ign http://ftp.us.debian.org sid/main 2014-07-09-0854.18.pdiff Get:4 http://ftp.us.debian.org sid/main Translation-da [2,414 kB] Get:5 http://ftp.us.debian.org sid/main Translation-es [323 kB] Get:6 http://ftp.us.debian.org sid/main Translation-pt [444 kB] Get:7 http://ftp.us.debian.org sid/main Translation-sk [421 kB] Get:8 http://ftp.us.debian.org sid/main Translation-uk [776 kB] Get:9 http://ftp.us.debian.org sid/main Translation-ja [786 kB] Get:10 http://ftp.us.debian.org sid/main Translation-de [1,772 kB] Get:11 http://ftp.us.debian.org sid/main Translation-it [3,259 kB] Get:12 http://ftp.us.debian.org sid/main Sources [7,333 kB] Get:13 http://ftp.us.debian.org sid/main amd64 Packages [6,820 kB] Get:14 http://ftp.us.debian.org sid/main [116 kB] Get:15 http://ftp.us.debian.org sid/main [35.0 kB] Fetched 26.2 MB in 1min 31s (288 kB/s) W: Failed to fetch xz:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_source_Sources Hash Sum mismatch W: Failed to fetch xz:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_binary-amd64_Packages Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-ru Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-pl Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-fr Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-da Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-es Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-pt Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-sk Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-uk Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-ja Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-de Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-it Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-fi Hash Sum mismatch W: Failed to fetch bzip2:/var/lib/apt/lists/partial/ftp.us.debian.org_debian_dists_sid_main_i18n_Translation-hu Hash Sum mismatch E: Some index files failed to download. They have been ignored, or old ones used instead. buildbot@debian-sid-amd64:~$ sudo apt-get dist-upgrade Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

2 2

[Maria-developers] [GSoC] Suggestions on adding a new method
by sriram patil 11 Jul '14

11 Jul '14

Hi All, I was thinking of adding two new inline methods in struct LEX as follows inline bool is_create_or_replace() { return (create_info.options & HA_LEX_CREATE_REPLACE); } inline bool is_create_if_not_exists() { return (create_info.options & HA_LEX_CREATE_IF_NOT_EXISTS); } With these, statements like if(lex->create_info.options & HA_LEX_CREATE_REPLACE) will be changed to if(lex->is_create_or_replace()) which definitely is more readable. This kind of statements are used in sql_yacc.yy, sql_parse.cc and many object specific implementation .cc files. Should I make the changes or is it unnecessary? Thanks, Sriram

1 0

Re: [Maria-developers] [JIRA] (MDEV-4937) NEED REVIEW: sql_slave_skip_counter does not work with GTID
by Kristian Nielsen 09 Jul '14

09 Jul '14

"Michael Widenius (JIRA)" <jira(a)mariadb.atlassian.net> writes: > Michael Widenius commented on MDEV-4937: > ---------------------------------------- Thanks for review Monty! I have some comments inline, let me know if you still want me to change something. - Kristian. > revno: 4260 > revision-id: knielsen at knielsen-hq.org-20140620104939-xdvn985cgmwy3odd > committer: Kristian Nielsen <knielsen at knielsen-hq.org> > timestamp: Fri 2014-06-20 12:49:39 +0200 > message: > MDEV-4937: sql_slave_skip_counter does not work with GTID > === modified file 'sql/slave.cc' > --- a/sql/slave.cc 2014-06-19 12:50:43 +0000 > +++ b/sql/slave.cc 2014-06-20 10:49:39 +0000 > @@ -3521,9 +3521,6 @@ static int exec_relay_log_event(THD* thd > > if (opt_gtid_ignore_duplicates) > { > - serial_rgi->current_gtid.domain_id= gev->domain_id; > - serial_rgi->current_gtid.server_id= gev->server_id; > - serial_rgi->current_gtid.seq_no= gev->seq_no; > int res= rpl_global_gtid_slave_state.check_duplicate_gtid > (&serial_rgi->current_gtid, serial_rgi); > if (res < 0) > What was the reason for removing the above assignments? > Was it a bug in the old code ? Not really a bug, however, just before this code we have: if (event_group_new_gtid(serial_rgi, gev)) and event_group_new_gtid() does the exact same assignments, so the ones I removed are completely redundant. I just happened to notice this redundancy, so I removed them... > (I have to ask as I am not 100% of what the old code did). > @@ -4594,16 +4598,27 @@ log '%s' at position %s, relay log '%s' > > if (saved_skip && rli->slave_skip_counter == 0) > { > + String tmp; > + if (mi->using_gtid != Master_info::USE_GTID_NO) > + { > + tmp.append(STRING_WITH_LEN(", GTID '")); > + rpl_append_gtid_state(&tmp, false); > + tmp.append(STRING_WITH_LEN("'; ")); > + } > > {noformat} > > If you want to avoid checking if tmp.append() fails, you could > create tmp with a static buffer. > > char tmp_buff[MAX_GTID_LENGTH+10]; > String tmp(tmp_buff, sizeof(tmp_buff), &my_charset_bin); > tmp.length(0); Right, hm. It does not really feel right to enforce a limit on the length; if the user uses a longer GTID position (which would be rather unusual), it might be confusing to have it silently cut. If the append() should fail, well, then it _will_ be cut, but then it's just an informational message, and that kind of out-of-memory is probably unlikely (and the error log will likely be filled up with other complaints about out-of-memory). But let me know if you still think I should change it. (If so - can you think of a place where MAX_GTID_LENGTH can be defined, that would be accessible to both client/mysqldump.c and sql/slave.cc? I could spot only my_global.h, which seems a bit too generic to contain GTID-related stuff...?) > if (saved_skip && rli->slave_skip_counter == 0) > { > String tmp; > saved_skip= 0; > + saved_skip_gtid_pos.free(); > } > If you free saved_skip_gtid_pos, it good to also free tmp! Right ... but it should be freed automatically anyway, when tmp goes out of scope, right? > === modified file 'sql/sys_vars.cc' > --- a/sql/sys_vars.cc 2014-06-09 18:00:23 +0000 > +++ b/sql/sys_vars.cc 2014-06-20 10:49:39 +0000 > @@ -4287,11 +4287,6 @@ bool update_multi_source_variable(sys_va > > static bool update_slave_skip_counter(sys_var *self, THD *thd, Master_info *mi) > { > - if (mi->using_gtid != Master_info::USE_GTID_NO) > - { > - my_error(ER_SLAVE_SKIP_NOT_IN_GTID, MYF(0)); > - return true; > - } > > {noformat} > > Shouldn't you also remove the error ER_SLAVE_SKIP_NOT_IN_GTID from the > error file? I think we cannot remove error codes from a GA release once introduced, right? So I left it, to not shift other error numbers. But let me know if I'm wrong, and I'll remove it. - Kristian.

1 0