January 2015 - developers - lists.mariadb.org

Re: [Maria-developers] [Commits] Rev 4562: MDEV-7402 'reset master' hangs, waits for signalled COND_xid_list in lp:~maria-captains/maria/10.0
by Kristian Nielsen 19 Jan '15

19 Jan '15

Sergei Golubchik <serg(a)mariadb.org> writes: > ------------------------------------------------------------ > revno: 4562 > revision-id: sergii(a)pisem.net-20150118130747-rs67yzzk8a7ebwdd > parent: sergii(a)pisem.net-20150118002113-snztyzwvzc500c2m > fixes bug: https://mariadb.atlassian.net/browse/MDEV-7402 > committer: Sergei Golubchik <sergii(a)pisem.net> > branch nick: 10.0 > timestamp: Sun 2015-01-18 14:07:47 +0100 > message: > MDEV-7402 'reset master' hangs, waits for signalled COND_xid_list > > Using a boolean flag for 'there is a RESET MASTER in progress' doesn't > work very well for multiple concurrent RESET MASTER statements. Ok to push with a testcase like the below, as we discussed. - Kristian. ----------------------------------------------------------------------- --echo *** MDEV-7402: 'reset master' hangs, waits for signalled COND_xid_list *** --source include/wait_for_binlog_checkpoint.inc connect(con3,localhost,root,,); # Make the binlog background thread wait before clearing the pending checkpoint. # The bug was that one RESET MASTER would clear the reset_master_pending # flag set by another RESET MASTER; this could cause the wakeup from the # binlog background thread not to be sent, and thus the second RESET MASTER # to wait infinitely. SET debug_sync="reset_logs_after_set_reset_master_pending SIGNAL reset_master_ready WAIT_FOR reset_master_cont"; send RESET MASTER; --connection default SET @old_dbug= @@global.DEBUG_DBUG; SET GLOBAL debug_dbug="+d,inject_binlog_background_thread_before_mark_xid_done"; SET debug_sync="now WAIT_FOR reset_master_ready"; RESET MASTER; SET debug_sync="now WAIT_FOR injected_binlog_background_thread"; SET GLOBAL debug_dbug=@old_dbug; SET debug_sync="now SIGNAL reset_master_cont"; --connection con3 REAP; --connection default SET debug_sync = 'reset';

1 0

Re: [Maria-developers] [MDEV-7362] ANALYZE TABLES crash with table-independent-statistics gathering
by Sergei Golubchik 17 Jan '15

17 Jan '15

Hi, Vicențiu! On Jan 17, Vicențiu Ciorbaru wrote: > > Here's the new diff, let me know if you have any more comments. Shall I > push this as is? Yes, just one small detail (file name and the comment), see below. Then please push, no need to ask for another review :) But... 1. Please don't paste patches into the email anymore. At least not with your mail client that wraps long lines around and sends everything as html. See https://mariadb.com/kb/en/using-git/#commit-emails 2. Why git? This is the patch for 10.0 bug, at the moment 10.0 is still in bzr. 3. This looks like a reverse diff, how did you manage to do that? :) > diff --git a/mysql-test/t/statistics_index_crash.test b/mysql-test/t/statistics_index_crash.test > deleted file mode 100644 > index dbbdca3..0000000 > --- a/mysql-test/t/statistics_index_crash.test if you create a new test file for one bug only (as opposite to adding to an existing test file), mention the bug number in the file name. Like statistics_index_crash-7362.test > +++ /dev/null > @@ -1,26 +0,0 @@ > ---source include/have_stat_tables.inc > ---source include/have_innodb.inc Start a test case for a bug with a comment that says the bug number, the synopsys, and optionally more information, if needed. The bare minimum looks like # # MDEV-7362 ANALYZE TABLES crash with table-independent-statistics gathering # MDEV-7380 engine-independent stats SEGV on ANALYZE TABLE (#2) # > - > -CREATE TABLE t1 (a longtext, FULLTEXT KEY (`a`)) ENGINE=InnoDB; > -INSERT INTO t1 VALUES (unhex('3E0D0A4141414142334E7A6143317963324541414141424977414141674541726D')); > -ANALYZE TABLE t1 PERSISTENT FOR ALL; > ---sorted_result > -SELECT * FROM mysql.index_stats WHERE index_name='a' AND table_name='t1'; > -DROP TABLE t1; > - > -CREATE TABLE t1 (a longtext, FULLTEXT KEY (`a`)) ENGINE=MyISAM; > -INSERT INTO t1 VALUES (unhex('3E0D0A4141414142334E7A6143317963324541414141424977414141674541726D')); > -ANALYZE TABLE t1 PERSISTENT FOR ALL; > ---sorted_result > -SELECT * FROM mysql.index_stats WHERE index_name='a' AND table_name='t1'; > -DROP TABLE t1; > - > -CREATE TABLE geom (g GEOMETRY NOT NULL, SPATIAL INDEX(g)) ENGINE=MyISAM; > -INSERT INTO geom VALUES > - (MultiPolygonFromText('MULTIPOLYGON(((28 26,28 0,84 0,84 42,28 26),(52 18,66 23,73 9,48 6,52 18)),((59 18,67 18,67 13,59 13,59 18)))')), > - (MPolyFromText('MULTIPOLYGON(((28 26,28 0,84 0,84 42,28 26),(52 18,66 23,73 9,48 6,52 18)),((59 18,67 18,67 13,59 13,59 18)))')), > - (MPolyFromWKB(AsWKB(MultiPolygon(Polygon(LineString(Point(0, 3), Point(3, 3), Point(3, 0), Point(0, 3))))))); > -ANALYZE TABLE geom PERSISTENT FOR ALL; > ---sorted_result > -SELECT * FROM mysql.index_stats WHERE index_name='g' AND table_name='geom'; > -DROP TABLE geom; Regards, Sergei

1 0

[Maria-developers] [MDEV-7362] ANALYZE TABLES crash with table-independent-statistics gathering
by Vicențiu Ciorbaru 17 Jan '15

17 Jan '15

Hi Sergei! I've implemented a fix for the bug described in MDEV-7362, that also fixes MDEV-7380. The test case also covers MDEV-7380. With the following fix,the I've spent a bit of time trying to figure out the architecture of things there. I am not confident my fix is complete, or if this is the way we want to go about fixing it. The fix does what we discussed over IRC, that we are not interested in computing statistics for a FULLTEXT index. What I think would be the root cause of the problem is calling ha_index_init() on an index that does not support this. By calling table->file->ha_index_init(), the table->field array gets corrupted and that ultimately leads to a crash in the function using it afterwards. Then again, this module is completely new to me, so any comments are appreciated. Here is the diff of the first changes. diff --git a/mysql-test/t/mdev-7362.test b/mysql-test/t/mdev-7362.test new file mode 100644 index 0000000..6551893 --- /dev/null +++ b/mysql-test/t/mdev-7362.test @@ -0,0 +1,14 @@ +CREATE TABLE t1 (a longtext, FULLTEXT KEY (`a`)) ENGINE=MyISAM; +insert into t1 values (unhex('3E0D0A4141414142334E7A6143317963324541414141424977414141674541726D')); +analyze table t1 persistent for all; +drop table t1; + +CREATE TABLE t1 (a longtext, FULLTEXT KEY (`a`)) ENGINE=MyISAM; +insert into t1 values (unhex('3E0D0A4141414142334E7A6143317963324541414141424977414141674541')); +analyze table t1 persistent for all; +drop table t1; + +CREATE TABLE t1 (f longtext NOT NULL, FULLTEXT KEY f (f)) ENGINE=MyISAM; +INSERT INTO t1 VALUES (REPEAT('a',27)),('foo'); +ANALYZE TABLE t1 PERSISTENT FOR ALL; +drop table t1; diff --git a/sql/sql_statistics.cc b/sql/sql_statistics.cc index d368145..2c1cccf 100644 --- a/sql/sql_statistics.cc +++ b/sql/sql_statistics.cc @@ -2367,6 +2367,9 @@ int collect_statistics_for_index(THD *thd, TABLE *table, uint index) DBUG_RETURN(rc); } + if (key_info->flags & HA_FULLTEXT) // skip if FULLTEXT index + DBUG_RETURN(rc); + table->key_read= 1; table->file->extra(HA_EXTRA_KEYREAD); Regards, Vicențiu

3 5

[Maria-developers] MDEV-7440 Build fails in libgroonga on Solaris
by Alexander Barkov 16 Jan '15

16 Jan '15

Hello Kentoku, Can you please have a look into this problem? https://mariadb.atlassian.net/browse/MDEV-7440 Can you fix please? Thanks.

1 0

Re: [Maria-developers] [Commits] Rev 4398: MDEV-7368 : SLES: Failed to start mysql.service: Unit
by Sergei Golubchik 15 Jan '15

15 Jan '15

Hi, Nirbhay! On Dec 23, Nirbhay Choubey wrote: > At lp:~maria-captains/maria/5.5 > > ------------------------------------------------------------ > revno: 4398 > revision-id: nirbhay(a)mariadb.com-20141224033207-8lh5i8d84qsjey79 > parent: elenst(a)montyprogram.com-20141223172123-pkfhjnv7oy1suxpa > committer: Nirbhay Choubey <nirbhay(a)mariadb.com> > branch nick: b7368-5.5 > timestamp: Tue 2014-12-23 22:32:07 -0500 > message: > MDEV-7368 : SLES: Failed to start mysql.service: Unit > mysql.service failed to load > > Added 'systemctl daemon-reload' command in postin and > postun rpm scripts. > === modified file 'support-files/rpm/server-postin.sh' > --- a/support-files/rpm/server-postin.sh 2013-11-20 11:18:46 +0000 > +++ b/support-files/rpm/server-postin.sh 2014-12-24 03:32:07 +0000 > @@ -1,6 +1,10 @@ > > # Make MySQL start/shutdown automatically when the machine does it. > if [ $1 = 1 ] ; then > + if [ -x /usr/bin/systemctl ] ; then > + /usr/bin/systemctl daemon-reload >/dev/null 2>&1 > + fi > + looks ok > if [ -x /sbin/chkconfig ] ; then > /sbin/chkconfig --add mysql > fi > @@ -51,30 +55,31 @@ > SETARGETDIR=/etc/selinux/targeted/src/policy > SEDOMPROG=$SETARGETDIR/domains/program > SECONPROG=$SETARGETDIR/file_contexts/program > +if [ -f /etc/redhat-release ] ; then > + if grep -q "Red Hat Enterprise Linux .. release 4" /etc/redhat-release \ > + || grep -q "CentOS release 4" /etc/redhat-release ; then I'd rather written if grep -Eqs '(Red Hat Enterprise Linux ..|CentOS) release 4' /etc/redhat-release; then or (which is more portable, but perhaps unnecessary so for an rpm postin script) if grep '$Red Hat Enterprise Linux ..\|CentOS$ release 4' /etc/redhat-release >/dev/null 2>&1; then but if you'd like you can push your variant. Regards, Sergei

2 1

Re: [Maria-developers] [Commits] Rev 4401: MDEV-7448 - mtr may leave stale mysqld in lp:maria/5.5
by Sergei Golubchik 14 Jan '15

14 Jan '15

Hi, Sergey! On Jan 14, Sergey Vojtovich wrote: > ------------------------------------------------------------ > revno: 4401 > revision-id: svoj(a)mariadb.org-20150114135038-jss2vvpkhihysfmj > parent: sergii(a)pisem.net-20150110130746-xlrofqddt4j8xkw6 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 5.5 > timestamp: Wed 2015-01-14 17:50:38 +0400 > message: > MDEV-7448 - mtr may leave stale mysqld > > Let mtr handle SIGHUP the same way as SIGINT. This solves stale mysqld processes > left after broken/closed ssh connection. Looks ok. But please make sure that all tests pass before pushing it. mysqld uses SIGHUP to print various information to the error log. If there's a test that actually tests that by sending SIGHUP - your change might've broken it. Regards, Sergei

1 0

[Maria-developers] MySQL bug #68557 "sjis changes in 5.5 break date parsing" fixed in MariaDB >= 5.5.38
by Hartmut Holzgraefe 13 Jan '15

13 Jan '15

Any idea how that happened, as I can't find anything that seems to be related in the 5.5.38 changelog? http://bugs.mysql.com/bug.php?id=68557 -- Hartmut Holzgraefe, Principal Support Engineer (EMEA) MariaDB Corporation | http://www.mariadb.com/

2 1

[Maria-developers] Buildbot slow upload
by Kristian Nielsen 13 Jan '15

13 Jan '15

Hi Serg, Daniel, Some time ago there was discussion of slow Buildbot upload speed (sorry, don't have the original mail thread to reply to). I found some new information on this. Yesterday I had to do a custom build, but was not able to because my tarbake upload kept getting stuck. So I had to investigate. Turns out the real problem was that hasky:/media/backup/archive/pack was full. And apparently, the error handling in Buildbot upload step is broken, so the error is not caught properly but just causes the upload to hang. I have noticed this before, that upload gets stuck. I wonder if full disk on hasky has caused this before. Daniel: Do we have something that removes old builds from /archive/pack? I did not see anything in root or buildbot crontabs? With respect to slow uploads, while trying various things, one thing I did was increase the upload blocksize in Buildbot config. Now I see tarbake upload (50MB) taking just 1 minute, which seems fair (probably it can be more if the network or buildbot is busy). So I'm wondering if the slow upload that was seen was in fact a combination of too low default blocksize, and uploads occasionally getting _completely_ stuck? It's of course still fine to move to using rsync, as Serg proposed earlier; just wanted to mention what I found. And in either case, we still need a solution to prevent /archive/pack from running full in the future... - Kristian.

2 1

[Maria-developers] MDEV-7288 USER/ROLE: CREATE OR REPLACE, CREATE IF NOT EXISTS, DROP IF EXISTS
by Alexander Barkov 09 Jan '15

09 Jan '15

Hi Sergei, Please have a look into the patch for MDEV-7288. It consists of the USER/ROLE related part of the big GSoC patch from Sriram that you reviewed earlier, with review suggestions addressed. You earlier had the following suggestions/questions: >> diff --git a/sql/sql_acl.cc b/sql/sql_acl.cc >> index 5f077c2..1e156be 100644 >> --- a/sql/sql_acl.cc >> +++ b/sql/sql_acl.cc >> ... >> @@ -9215,10 +9252,26 @@ bool mysql_create_user(THD *thd, List <LEX_USER> &lis >> */ >> if (handle_grant_data(tables, 0, user_name, NULL)) >> { >> - append_user(thd, &wrong_users, user_name); >> - >> - result= TRUE; >> - continue; >> + if (thd->lex->is_create_or_replace()) >> + { >> + if (handle_grant_data(tables, 1, user_name, NULL) <= 0) >> + { >> + append_user(thd, &wrong_users, user_name); > > I don't understand that at all. If user creation failed, you try to drop > it, but don't create a user again, after the old one was dropped. How > could it work? > Yeah, it looks confusing, because handle_grant_data() can do different things depending on the arguments. The call for handle_grant_data(tables, 0, user_name, NULL) does not create any users. It only checks if the user already exists. The call for handle_grant_data(tables, 1, user_name, NULL) drops the user. And the call for replace_user_table() after this block is the one that actually creates the user. I have added some comments in the new code, to make it clearer. >>> @@ -9268,7 +9321,7 @@ bool mysql_create_user(THD *thd, List <LEX_USER> &list, >>> (handle_as_role) ? "CREATE ROLE" : "CREATE USER", >>> wrong_users.c_ptr_safe()); >>> >>> - if (some_users_created) >>> + if (some_users_created || !result) >> >> What is that for? To binlog CREATE IF NOT EXISTS? Yes, that was the reason. I renamed "some_users_created" to "binlog" and changed the logic slightly to set "binlog" to true under the block checking lex->create_info.if_not_exists(), instead of this additional test for "result". So now it should be easier to read: if (binlog) result |= write_bin_log(thd, FALSE, thd->query(), thd->query_length()); Thanks.

3 3

[Maria-developers] prefix matching for command-line options - how to disable?
by Sergei Golubchik 06 Jan '15

06 Jan '15

Hi, We've got a bit of a problem with command-line option prefixes in the test suite. As you know, in MariaDB a command line option (the same applies to my.cnf too) doesn't need to be specified completely, it's sufficient to use an unambigous prefix of it. For example, 'net-retry-count=5' in my.cnf can be abbreviated as 'net-ret=5'. But not as 'net-re=5', the latter will fail with an error mysqld: ambiguous option '--net-re' (net_read_timeout, net_retry_count) This is the historical behavior of MySQL since about 2002. But it creates problems for plugins. InnoDB is particularly badly affected - it has lots of I_S plugins that are prefixes of other I_S plugins. For example: INNODB_BUFFER_PAGE INNODB_BUFFER_PAGE_LRU ... INNODB_CMP INNODB_CMPMEM INNODB_CMPMEM_RESET INNODB_CMP_PER_INDEX INNODB_CMP_PER_INDEX_RESET INNODB_CMP_RESET ... INNODB_SYS_FOREIGN INNODB_SYS_FOREIGN_COLS ... INNODB_SYS_TABLES INNODB_SYS_TABLESPACES INNODB_SYS_TABLESTATS also, it has plugins that match other plugin variables, like INNODB_LOCKS and innodb_locks_unsafe_for_binlog. As a result one cannot see or predict what these options will do. For example --enable-innodb-buffer-page might enable INNODB_BUFFER_PAGE plugin. Or it might enable INNODB_BUFFER_PAGE_LRU plugin - depending on which plugin will loaded (and will see and consume the option) first. And one cannot possibly disable INNODB_BUFFER_PAGE plugin, if the server will try to load INNODB_BUFFER_PAGE_LRU first. I see few solutions for this issue: 1. support a special suffix to option names that will disable prefix matching, for example: mysqld --enable-innodb-buffer-page! this will not match innodb-buffer-page-lru, so it's unambiguous. Of course, any dedicated character can be used, --enable-innodb-buffer-page$, or even a prefix --enable-exact-innodb-buffer-page, --enable-strict-innodb-buffer-page 2. add a special command-line option to disable prefix matching: mysqld --disable-getopt-prefix-matching --enable-innodb-buffer-page 3. do like mysql-5.7 is doing and completely remove support for prefix matching. In 5.7 one has to specified option names in full. I suspect that this is likely to break many exising setups. Opinions? Anybody cares about this obscure issue at all? Regards, Sergei

3 3