- developers - lists.mariadb.org

Re: [Maria-developers] [Commits] Rev 4056: MDEV-5607: Query cache destroys uninitialized rwlock in file:///home/bell/maria/bzr/work-maria-5.5-MDEV-5607/
by Sergey Vojtovich 10 Feb '14

10 Feb '14

Hi Sanja, JFYI: while reporting this bug I came up with two questions that I couldn't answer quickly. Probably they make sense, if not just ignore them: 1. This code was introduced with https://bugs.launchpad.net/maria/+bug/782223, revision: wlad(a)montyprogram.com-20110514163720-rmico2qegtptjguk <quot> The callstack leading of "free" containing critical section is: mysqld!free my_no_flags_free Query_cache::free_cache Query_cache::resize fix_query_cache_size set_var::update sql_set_variables mysql_execute_command mysql_parse <quot> It means that this code was supposed to be executed exactly by Query_cache::resize(), where you disable it. Question: if this code is not supposed to be executed by ::resize(), is it needed at all? 2. In Query_cache::resize() I can see block-level locks are acquired with the following comment: <quot> Wait for all readers and writers to exit. When the list of all queries is iterated over with a block level lock, we are done. </quot> Isn't it needed in ::free_cache() also? Regards, Sergey On Sat, Feb 08, 2014 at 01:25:11PM +0000, sanja(a)askmonty.org wrote: > At file:///home/bell/maria/bzr/work-maria-5.5-MDEV-5607/ > > ------------------------------------------------------------ > revno: 4056 > revision-id: sanja(a)askmonty.org-20140208132448-qdkkpdncnya0tr6j > parent: elenst(a)wheezy-64.home-20140205102537-7ern5gfca6a6ojg3 > committer: sanja(a)askmonty.org > branch nick: work-maria-5.5-MDEV-5607 > timestamp: Sat 2014-02-08 15:24:48 +0200 > message: > MDEV-5607: Query cache destroys uninitialized rwlock > > - Resize destroyed rw_lock twice, now it is prevented. > - Trash destroyed rw lock. > === modified file 'include/my_valgrind.h' > --- a/include/my_valgrind.h 2012-03-22 11:31:09 +0000 > +++ b/include/my_valgrind.h 2014-02-08 13:24:48 +0000 > @@ -13,7 +13,8 @@ > along with this program; if not, write to the Free Software > Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */ > > - > +#ifndef my_valgrind_h > +#define my_valgrind_h > /* Some defines to make it easier to use valgrind */ > #include <m_string.h> /* bfill */ > > @@ -45,3 +46,4 @@ > #define TRASH_FREE(A,B) TRASH_FILL(A,B,0x8F) > #define TRASH(A,B) TRASH_FREE(A,B) > > +#endif //my_valgrind_h > > === modified file 'include/mysql/psi/mysql_thread.h' > --- a/include/mysql/psi/mysql_thread.h 2013-01-11 00:03:43 +0000 > +++ b/include/mysql/psi/mysql_thread.h 2014-02-08 13:24:48 +0000 > @@ -16,6 +16,8 @@ > #ifndef MYSQL_THREAD_H > #define MYSQL_THREAD_H > > +#include <my_valgrind.h> > + > /** > @file mysql/psi/mysql_thread.h > Instrumentation helpers for mysys threads, mutexes, > @@ -714,6 +716,7 @@ static inline int inline_mysql_prlock_in > static inline int inline_mysql_rwlock_destroy( > mysql_rwlock_t *that) > { > + int res; > #ifdef HAVE_PSI_INTERFACE > if (likely(PSI_server && that->m_psi)) > { > @@ -721,7 +724,9 @@ static inline int inline_mysql_rwlock_de > that->m_psi= NULL; > } > #endif > - return rwlock_destroy(&that->m_rwlock); > + res= rwlock_destroy(&that->m_rwlock); > + TRASH(that, sizeof(*that)); > + return res; > } > > #ifndef DISABLE_MYSQL_PRLOCK_H > > === modified file 'mysql-test/r/query_cache.result' > --- a/mysql-test/r/query_cache.result 2013-10-16 13:07:25 +0000 > +++ b/mysql-test/r/query_cache.result 2014-02-08 13:24:48 +0000 > @@ -2077,6 +2077,21 @@ drop procedure p1; > drop table t1; > set GLOBAL query_cache_size=1355776; > SET GLOBAL userstat=default; > +# > +#MDEV-5607 Query cache destroys uninitialized rwlock > +# > +SET @global_query_cache_size = @@global.query_cache_size; > +SET @global_query_cache_type = @@global.query_cache_type; > +SET GLOBAL query_cache_type = default; > +SET GLOBAL query_cache_size = default; > +SET GLOBAL query_cache_type = ON; > +SET GLOBAL query_cache_size = 131072; > +CREATE TABLE t1(a INT); > +SELECT * FROM t1; > +a > +SET GLOBAL query_cache_size = @global_query_cache_size; > +SET GLOBAL query_cache_type = @global_query_cache_type; > +DROP TABLE t1; > End of 5.5 tests > restore defaults > SET GLOBAL query_cache_type= default; > > === modified file 'mysql-test/t/query_cache.test' > --- a/mysql-test/t/query_cache.test 2013-10-16 13:07:25 +0000 > +++ b/mysql-test/t/query_cache.test 2014-02-08 13:24:48 +0000 > @@ -1708,6 +1708,23 @@ drop table t1; > set GLOBAL query_cache_size=1355776; > SET GLOBAL userstat=default; > > +--echo # > +--echo #MDEV-5607 Query cache destroys uninitialized rwlock > +--echo # > +SET @global_query_cache_size = @@global.query_cache_size; > +SET @global_query_cache_type = @@global.query_cache_type; > +SET GLOBAL query_cache_type = default; > +SET GLOBAL query_cache_size = default; > +SET GLOBAL query_cache_type = ON; > +SET GLOBAL query_cache_size = 131072; > + > +CREATE TABLE t1(a INT); > +SELECT * FROM t1; > + > +SET GLOBAL query_cache_size = @global_query_cache_size; > +SET GLOBAL query_cache_type = @global_query_cache_type; > +DROP TABLE t1; > + > --echo End of 5.5 tests > > --echo restore defaults > > === modified file 'sql/sql_cache.cc' > --- a/sql/sql_cache.cc 2013-11-19 12:16:25 +0000 > +++ b/sql/sql_cache.cc 2014-02-08 13:24:48 +0000 > @@ -1326,6 +1326,7 @@ ulong Query_cache::resize(ulong query_ca > query->unlock_n_destroy(); > block= block->next; > } while (block != queries_blocks); > + queries_blocks= NULL; // avoid second destroying by free_cache > } > free_cache(); > > > === modified file 'storage/perfschema/ha_perfschema.cc' > --- a/storage/perfschema/ha_perfschema.cc 2012-03-27 23:04:46 +0000 > +++ b/storage/perfschema/ha_perfschema.cc 2014-02-08 13:24:48 +0000 > @@ -19,9 +19,9 @@ > */ > > #include "my_global.h" > +#include "sql_plugin.h" > #include "my_pthread.h" > #include "my_atomic.h" > -#include "sql_plugin.h" > #include "mysql/plugin.h" > #include "ha_perfschema.h" > #include "pfs_engine_table.h" > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits

2 1

[Maria-developers] Buildbot trees for spider and mroonga
by Alexander Barkov 10 Feb '14

10 Feb '14

Hi Kentoku, Currently you use your own trees for Mroonga and Spider development, e.g. bzr+ssh://bazaar.launchpad.net/~mroonga/maria/mroonga/ Would you mind to move development into trees under ~maria-captains location, for example: bzr+ssh://bazaar.launchpad.net/~maria-captains/maria/10.0-mroonga/ bzr+ssh://bazaar.launchpad.net/~maria-captains/maria/10.0-spider/ This will give an great advantage to have every revision compiled and tested on a wide variety of operating systems (this is done automatically by MariaDB Build Bot), and also will help us to merge your updates into the main 10.0 tree easier. Note, development in these trees should be done "incrementally". Things like "bzr push --overwrite" are not desirable, to avoid problems with BiildBot and with merges. Hope this is not a big problem. Thanks.

3 4

[Maria-developers] Updating OQGRAPH tables
by Ian Gilfillan 08 Feb '14

08 Feb '14

Hi I have been struggling to get OQGRAPH working with MariaDB 10.0.7. The system has been hanging and crashing frequently. I will post as bugs if I can recreate them, but this question is simply, how can I update the data? On a new install of MariaDB 10.0.7 and OQGRAPH - I create the sample tables at https://mariadb.com/kb/en/oqgraph-examples/#creating-a-table-with-weight - I then update the oq2_backing table. The changes reflect on the oq2_backing table, not the oq2_graph table: UPDATE oq2_backing SET weight=4 WHERE origid=2 AND destid=6; SELECT * FROM oq2_backing; +--------+--------+--------+ | origid | destid | weight | +--------+--------+--------+ | 1 | 2 | 1 | | 2 | 3 | 1 | | 2 | 6 | 4 | | 3 | 4 | 3 | | 4 | 5 | 1 | | 5 | 6 | 2 | +--------+--------+--------+ SELECT * FROM oq2_graph; +-------+--------+--------+--------+------+--------+ | latch | origid | destid | weight | seq | linkid | +-------+--------+--------+--------+------+--------+ | NULL | 1 | 2 | 1 | NULL | NULL | | NULL | 2 | 3 | 1 | NULL | NULL | | NULL | 2 | 6 | 10 | NULL | NULL | | NULL | 3 | 4 | 3 | NULL | NULL | | NULL | 4 | 5 | 1 | NULL | NULL | | NULL | 5 | 6 | 2 | NULL | NULL | +-------+--------+--------+--------+------+--------+ How can I make changes that reflect on the table? thanks, ian

2 2

[Maria-developers] GCC native thread local storage (Was: [Maria-discuss] xtradb in 10.0)
by Kristian Nielsen 07 Feb '14

07 Feb '14

Laurynas Biveinis <laurynas.biveinis(a)gmail.com> writes: > We have another platform-specific addition: thread-local storage > implemented by __thread GCC keyword, which is GNU specific. This is Hm, this is interesting, I was not aware of the __thread feature. This looks to be much more efficient than pthread_getspecific(), which is one of the top functions in the profiles for sysbench readonly: static __thread long tcount = 0; long bump(long del) { return (tcount += del); } 0000000000000000 <bump>: 0: 48 89 f8 mov %rdi,%rax 3: 64 48 03 04 25 00 00 add %fs:0x0,%rax a: 00 00 c: 64 48 89 04 25 00 00 mov %rax,%fs:0x0 13: 00 00 15: c3 retq It is just a single instruction using %fs: addressing to access the thread_local storage. Apparently the linker resolves the %fs: offset in the resulting binary: 0000000000400580 <bump>: 400580: 48 89 f8 mov %rdi,%rax 400583: 64 48 03 04 25 e0 ff add %fs:0xffffffffffffffe0,%rax 40058a: ff ff 40058c: 64 48 89 04 25 e0 ff mov %rax,%fs:0xffffffffffffffe0 400593: ff ff 400595: c3 retq 00000000004005a0 <bump2>: 4005a0: 48 89 f8 mov %rdi,%rax 4005a3: 64 48 03 04 25 f0 ff add %fs:0xfffffffffffffff0,%rax 4005aa: ff ff 4005ac: 64 48 89 04 25 f0 ff mov %rax,%fs:0xfffffffffffffff0 4005b3: ff ff 4005b5: c3 retq So that could be very promising for an easy way to tweak out a little more performance. Something to look into at some point. - Kristian.

1 0

[Maria-developers] MariaDB 10.0.8 release prep
by Daniel Bartholomew 06 Feb '14

06 Feb '14

I've started preparing for the MariaDB 10.0.8 release. Draft changelog and release notes are here: - https://mariadb.com/kb/en/mariadb-1008-changelog/ - https://mariadb.com/kb/en/mariadb-1008-release-notes/ Please let me know ASAP about things that should be mentioned in the release notes (or add them yourself). Thanks! -- Daniel Bartholomew MariaDB

1 0

Re: [Maria-developers] [Commits] Rev 3984: MDEV-4439 ALTER TABLE .. [ADD|DROP] FOREIGN KEY IF [NOT] EXISTS does not work if constraint name is not used. in file:///home/hf/wmar/10-hf/
by Jan Lindström 06 Feb '14

06 Feb '14

Hi, Change: Another change is that it uses the index name for the internal dictionary. Not ok. Firstly, that change should affect many test cases using foreign keys. Secondly, that would effect applications using foreign keys. I could only imagine how many and what kind of applications there are to create/drop foreign keys that are currently created with InnoDB internal foreign key names. Finally, this fix is 'correct' only for new tables, not existing tables containing foreign keys. This change would be incompatibility change to default behavior. > At file:///home/hf/wmar/10-hf/ > > ------------------------------------------------------------ > revno: 3984 > revision-id: holyfoot(a)askmonty.org-20140205184909-9nmsj1mzwo6cka0m > parent: sergii(a)pisem.net-20140205171851-0vax8lnwtbousucu > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: 10-hf > timestamp: Wed 2014-02-05 22:49:09 +0400 > message: > MDEV-4439 ALTER TABLE .. [ADD|DROP] FOREIGN KEY IF [NOT] EXISTS does not work if constraint name is not used. > Patches for server and the Innodb engine. > Server is fixed so it does nothing if no indexes left to alter. > Innodb parser is fixed so it looks for the IF [NOT] EXISTS option in a string. > Another change is that it uses the index name for the internal dictionary. > Prior to that it only used the CONSTRAINT name for it. > > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- -- Jan Lindström Principal Engineer MariaDB | MaxScale | skype: jan_p_lindstrom www.skysql.com <http://www.skysql.com/> Twitter <http://twitter.com/skysql> Blog <http://www.skysql.com/blog/> Facebook <http://www.facebook.com/skysql> LinkedIn <http://www.linkedin.com/company/1214250> Google+ <https://plus.google.com/117544963211695643458/posts>

2 3

Re: [Maria-developers] MariaDB
by Kristian Nielsen 04 Feb '14

04 Feb '14

Otto Kekäläinen <otto(a)seravo.fi> writes: > 2013-10-01 Otto Kekäläinen <otto(a)fsfe.org>: >> 2013/10/1 Ondřej Surý <ondrej(a)sury.org>: >>> - it would be better to repack the tarball and kill the upstream debian/ >>> directory to not mess with our packaging >> >> I hope to merge my work upstream and that is easier if my changes are >> a made on top of current upstream. I think most of the changes should >> be useful for upstream too. Maybe you can participate in the process >> to create patches and test that they work for upstream OK? > there much easier. Although I will be needing help from somebody to > make autobuild.sh and standard Debian builders work nicely on the same > code base. Wouldn't it be better to discard the existing debian/ stuff in the upstream sources, and work from the native Debian packaging as the base? The existing stuff has _so_ much legacy and crap, and little of value. It has origins in some ancient copy of native Debian packages, then OurDelta hacked it for their build infrastructure (the autobuild.sh stuff), then MariaDB for our build infrastructure. It has desperately needed a cleanup for so long. Since this cleanup has now been done in native Debian packaging, which also makes unnecessary some of the hacks to work with native Debian packages, it seems better to just use Otto's work as the base going forwards. And in any case, the old reason for having debian packaging in-tree no longer applies. Now, we need to make sure before each release of MariaDB that it will play nicely with the upstream Debian packaging, and for this it seems most useful to pull in the packaging from the Debian packaging git repo? Well, in the end it is up to you. I suppose in any case there will be a wish to provide 3rd-party MariaDB .debs, eg. to provide backports to older distros of new versions. It just seems more natural to focus primarily on having good native packages and then accommodating the 3rd-party stuff, rather than the other way around? - Kristian.

3 4

[Maria-developers] MDEV-5273
by Alexey Botchkov 04 Feb '14

04 Feb '14

Hi, Serg, all. I'm not sure how about fixing this one: https://mariadb.atlassian.net/browse/MDEV-5273 So would like your opinion. The thing is that the SHOW commands with old-fasioned implementations (those that aren't mapped to the SELECT FROM information_schema.something) return stmt->field_count == 0 after the mysql_prepare_statement. The fix i see involves some boring coding. Is it worth to get into it right now? Best regards. HF

2 1

Re: [Maria-developers] [Commits] Rev 3966: MDEV-4439 ALTER TABLE .. [ADD|DROP] FOREIGN KEY IF [NOT] EXISTS does not work if constraint name is not used. in file:///home/hf/wmar/10-hf/
by Jan Lindström 04 Feb '14

04 Feb '14

Hi, In my opinion the test case is not complete, you are missing e.g. ALTER TABLE t2 ADD FOREIGN KEY IF NOT EXISTS (id) REFERENCES t1(id); -- should fail on warning ALTER TABLE t2 ADD FOREIGN KEY IF NOT EXISTS (id) REFERENCES t1(id); ALTER TABLE t2 ADD FOREIGN KEY IF NOT EXISTS a(id REFERENCES t1(id); -- should fail, there is already foreign key for id, yes it name is not b or a but anyway ALTER TABLE t2 ADD FOREIGN KEY IF NOT EXISTS b(id) REFERENCES t1(id); R: > At file:///home/hf/wmar/10-hf/ > > ------------------------------------------------------------ > revno: 3966 > revision-id: holyfoot(a)askmonty.org-20140201114304-7bi0ug2yg3eyyhlr > parent: psergey(a)askmonty.org-20140121100700-9g8hl8vx8nmk1e2w > committer: Alexey Botchkov <holyfoot(a)askmonty.org> > branch nick: 10-hf > timestamp: Sat 2014-02-01 15:43:04 +0400 > message: > MDEV-4439 ALTER TABLE .. [ADD|DROP] FOREIGN KEY IF [NOT] EXISTS does not work if constraint name is not used. > Patches for server and the Innodb engine. > Server is fixed so it does nothing if no indexes left to alter. > Innodb parser is fixed so it looks for the IF [NOT] EXISTS option in a string. > Another change is that it uses the index name for the internal dictionary. > Prior to that it only used the CONSTRAINT name for it. > > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- -- Jan Lindström Principal Engineer MariaDB | MaxScale | skype: jan_p_lindstrom www.skysql.com <http://www.skysql.com/> Twitter <http://twitter.com/skysql> Blog <http://www.skysql.com/blog/> Facebook <http://www.facebook.com/skysql> LinkedIn <http://www.linkedin.com/company/1214250> Google+ <https://plus.google.com/117544963211695643458/posts>

2 3

[Maria-developers] Extra copying in debian/rules causes troubles
by Elena Stepanova 04 Feb '14

04 Feb '14

Hi all. There is a fragment in MariaDB debian/dist/*/rules files: ... ifeq ($(findstring nocheck,$(DEB_BUILD_OPTIONS)),) # Don't know why the following is necessary... cp unittest/unit.pl $(builddir)/unittest/ cp -r mysql-test/* $(builddir)/mysql-test/ cp -r sql/share/* $(builddir)/sql/share/ cp -r scripts/*sql $(builddir)/scripts/ if [ ! -f testsuite-stamp ] ; then \ ... Recently it started causing a problem. MTR assumes that the tests are run either in a proper source build (be it in-source or out-of-source), or in a binary build. With the extra-copying above, particularly due to copying of mysql-test folder, the builddir by the moment of running tests is in the "neither fish nor fowl" state. So if one runs dpkg-buildpackage with the tests enabled, it fails when it comes to the MTR stage. We had this problem reported at least 3 times over the last 24 hours, so it's not hypothetical. It's currently filed as https://mariadb.atlassian.net/browse/MDEV-5591. I tried to remove these 4 'cp' lines and the build seemed to work all right; but there might also be consequences that I fail to predict, so I would like to have the 2nd (3rd, 4th, ... Nth) opinion. Does anyone have any idea what might go wrong without these 4 lines? Also, *Kristian*, These lines are coming from the initial draft of the rules. I understand from the comment that there is no reason to ask *why* the lines were necessary, but I'm wondering if you remember why you felt they *were* necessary (even if didn't know why). Were you getting some errors without them? Or were they copied from some previous version of the file? I realize it was long time ago so you probably won't remember, but I thought it was worth checking anyway. Regards, Elena

3 4