- developers - lists.mariadb.org

[Maria-developers] Fwd: possible bug in MySQL 5.6.7-rc slave replication crash safety?
by Zardosht Kasheff 24 Oct '12

24 Oct '12

Hello, Does Maria 10.0 implement crash safe replication that is found in MySQL 5.6.7? If so, does this issue exist there? Thanks -Zardosht ---------- Forwarded message ---------- From: Zardosht Kasheff <zardosht(a)gmail.com> Date: Wed, Oct 17, 2012 at 8:59 AM Subject: possible bug in MySQL 5.6.7-rc slave replication crash safety? To: internals(a)lists.mysql.com Hello all, We downloaded MySQL 5.6.7-rc to see slave replication crash safety in action. A simple test did not seem to work, as described in http://bugs.mysql.com/bug.php?id=67246. Does anyone have insight into the problem? Perhaps we screwed something up? Thanks -Zardosht

2 8

Re: [Maria-developers] Please look at and advise about MDEV-3801
by Igor Babaev 22 Oct '12

22 Oct '12

On 10/22/2012 12:53 AM, Timour Katchaounov wrote: > Igor, > > Great, the patch indeed fixes the bug both on 5.3 and 5.5, however the > reduced test case I produced still takes 21 MB, and I was not successful > in reducing it to a size that can be added to the regression test suite. > I already spent too much in reducing the test case. Would you agree that > I push your patch without a test case? I think you still have to add the test case case (in a separate file) that is run with the --big option. Regards, Igor. > > Timour > > > On 20.10.2012 08:56, Igor Babaev wrote: >> On 10/19/2012 12:43 PM, Timour Katchaounov wrote: >>> Hi Igor, >>> >>> Could you please look at this bug >>> https://mariadb.atlassian.net/browse/MDEV-3801, >>> and read my last comment. The bug may be (just a guess ATM) related to >>> index >>> creation on the temporary table created for the derived table. Since >>> this is your >>> code, I'd like you to tell me if you have seen anything similar before, >>> or if >>> you have any hints. >>> >>> If not, I will continue with the bug and will trace the cause for the >>> wrong key information. >>> >>> Thanks, >>> Timour >> >> Timour, >> >> The following patch fixes this problem >> >> Regards, >> Igor. >> >> === modified file 'sql/sql_select.cc' >> --- sql/sql_select.cc 2012-10-10 05:21:22 +0000 >> +++ sql/sql_select.cc 2012-10-20 05:49:13 +0000 >> @@ -8981,7 +8981,7 @@ void JOIN::drop_unused_derived_keys() >> JOIN_TAB *tab; >> for (tab= first_linear_tab(this, WITHOUT_CONST_TABLES); >> tab; >> - tab= next_linear_tab(this, tab, WITHOUT_BUSH_ROOTS)) >> + tab= next_linear_tab(this, tab, WITH_BUSH_ROOTS)) >> { >> >> TABLE *table=tab->table; >> >> MariaDB [test]> set @@tmp_table_size=16384; >> ERROR 2006 (HY000): MySQL server has gone away >> No connection. Trying to reconnect... >> Connection id: 1 >> Current database: test >> >> Query OK, 0 rows affected (5.48 sec) >> >> MariaDB [test]> set @@tmp_table_size=16384; >> Query OK, 0 rows affected (0.00 sec) >> >> MariaDB [test]> EXPLAIN SELECT t1.deal_id FROM table_e AS t1 JOIN >> table_d AS t10 ON (t1.deal_id = t10.deal_id) WHERE t1.deal_id IN >> (SELECT * FROM (SELECT DISTINCT t1.deal_id FROM table_e AS >> t1 JOIN table_d AS t2 ON (t1.deal_id = t2.deal_id) >> JOIN table_c_s AS t3 ON (t2.deal_id = t3.deal_id) AND (t2.feid = >> t3.feid) JOIN table_f AS t4 ON t3.exchange = t4.exchange) >> tU) limit 10; >> +----+--------------+-------------+--------+-------------------------------------------+----------+---------+------------------------------+------+-----------------------+ >> >> | id | select_type | table | type | possible_keys >> | key | key_len | ref | >> rows | Extra | >> +----+--------------+-------------+--------+-------------------------------------------+----------+---------+------------------------------+------+-----------------------+ >> >> | 1 | PRIMARY | <subquery2> | ALL | distinct_key >> | NULL | NULL | NULL | >> 8488 | | >> | 1 | PRIMARY | t1 | eq_ref | PRIMARY >> | PRIMARY | 4 | tU.deal_id | >> 1 | Using index | >> | 1 | PRIMARY | t10 | ref | PRIMARY,full_id >> | PRIMARY | 4 | tU.deal_id | >> 1 | Using index | >> | 2 | MATERIALIZED | <derived3> | ALL | NULL >> | NULL | NULL | NULL | >> 8488 | | >> | 3 | DERIVED | t4 | system | PRIMARY >> | NULL | NULL | NULL | >> 1 | Using temporary | >> | 3 | DERIVED | t3 | ref | >> PRIMARY,feid,exchange,deal_feid,feid_date | exchange | 3 | const >> | 8488 | | >> | 3 | DERIVED | t1 | eq_ref | PRIMARY >> | PRIMARY | 4 | test.t3.deal_id | >> 1 | Using index | >> | 3 | DERIVED | t2 | eq_ref | PRIMARY,full_id >> | PRIMARY | 8 | test.t3.deal_id,test.t3.feid | >> 1 | Using index; Distinct | >> +----+--------------+-------------+--------+-------------------------------------------+----------+---------+------------------------------+------+-----------------------+ >> >> 8 rows in set (4.22 sec) >> >> MariaDB [test]> SELECT t1.deal_id FROM table_e AS t1 JOIN table_d AS t10 >> ON (t1.deal_id = t10.deal_id) WHERE t1.deal_id IN (SELECT * FROM >> (SELECT DISTINCT t1.deal_id FROM table_e AS t1 >> JOIN table_d AS t2 ON (t1.deal_id = t2.deal_id) JOIN >> table_c_s AS t3 ON (t2.deal_id = t3.deal_id) AND (t2.feid = t3.feid) >> JOIN table_f AS t4 ON t3.exchange = t4.exchange) tU) limit >> 10; >> +---------+ >> | deal_id | >> +---------+ >> | 30 | >> | 82 | >> | 83 | >> | 108 | >> | 155 | >> | 156 | >> | 162 | >> | 184 | >> | 186 | >> | 195 | >> +---------+ >> 10 rows in set (10.62 sec) >>

1 1

Re: [Maria-developers] Please look at and advise about MDEV-3801
by Igor Babaev 20 Oct '12

20 Oct '12

On 10/19/2012 12:43 PM, Timour Katchaounov wrote: > Hi Igor, > > Could you please look at this bug > https://mariadb.atlassian.net/browse/MDEV-3801, > and read my last comment. The bug may be (just a guess ATM) related to > index > creation on the temporary table created for the derived table. Since > this is your > code, I'd like you to tell me if you have seen anything similar before, > or if > you have any hints. > > If not, I will continue with the bug and will trace the cause for the > wrong key information. > > Thanks, > Timour Timour, The following patch fixes this problem Regards, Igor. === modified file 'sql/sql_select.cc' --- sql/sql_select.cc 2012-10-10 05:21:22 +0000 +++ sql/sql_select.cc 2012-10-20 05:49:13 +0000 @@ -8981,7 +8981,7 @@ void JOIN::drop_unused_derived_keys() JOIN_TAB *tab; for (tab= first_linear_tab(this, WITHOUT_CONST_TABLES); tab; - tab= next_linear_tab(this, tab, WITHOUT_BUSH_ROOTS)) + tab= next_linear_tab(this, tab, WITH_BUSH_ROOTS)) { TABLE *table=tab->table; MariaDB [test]> set @@tmp_table_size=16384; ERROR 2006 (HY000): MySQL server has gone away No connection. Trying to reconnect... Connection id: 1 Current database: test Query OK, 0 rows affected (5.48 sec) MariaDB [test]> set @@tmp_table_size=16384; Query OK, 0 rows affected (0.00 sec) MariaDB [test]> EXPLAIN SELECT t1.deal_id FROM table_e AS t1 JOIN table_d AS t10 ON (t1.deal_id = t10.deal_id) WHERE t1.deal_id IN (SELECT * FROM (SELECT DISTINCT t1.deal_id FROM table_e AS t1 JOIN table_d AS t2 ON (t1.deal_id = t2.deal_id) JOIN table_c_s AS t3 ON (t2.deal_id = t3.deal_id) AND (t2.feid = t3.feid) JOIN table_f AS t4 ON t3.exchange = t4.exchange) tU) limit 10; +----+--------------+-------------+--------+-------------------------------------------+----------+---------+------------------------------+------+-----------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+--------------+-------------+--------+-------------------------------------------+----------+---------+------------------------------+------+-----------------------+ | 1 | PRIMARY | <subquery2> | ALL | distinct_key | NULL | NULL | NULL | 8488 | | | 1 | PRIMARY | t1 | eq_ref | PRIMARY | PRIMARY | 4 | tU.deal_id | 1 | Using index | | 1 | PRIMARY | t10 | ref | PRIMARY,full_id | PRIMARY | 4 | tU.deal_id | 1 | Using index | | 2 | MATERIALIZED | <derived3> | ALL | NULL | NULL | NULL | NULL | 8488 | | | 3 | DERIVED | t4 | system | PRIMARY | NULL | NULL | NULL | 1 | Using temporary | | 3 | DERIVED | t3 | ref | PRIMARY,feid,exchange,deal_feid,feid_date | exchange | 3 | const | 8488 | | | 3 | DERIVED | t1 | eq_ref | PRIMARY | PRIMARY | 4 | test.t3.deal_id | 1 | Using index | | 3 | DERIVED | t2 | eq_ref | PRIMARY,full_id | PRIMARY | 8 | test.t3.deal_id,test.t3.feid | 1 | Using index; Distinct | +----+--------------+-------------+--------+-------------------------------------------+----------+---------+------------------------------+------+-----------------------+ 8 rows in set (4.22 sec) MariaDB [test]> SELECT t1.deal_id FROM table_e AS t1 JOIN table_d AS t10 ON (t1.deal_id = t10.deal_id) WHERE t1.deal_id IN (SELECT * FROM (SELECT DISTINCT t1.deal_id FROM table_e AS t1 JOIN table_d AS t2 ON (t1.deal_id = t2.deal_id) JOIN table_c_s AS t3 ON (t2.deal_id = t3.deal_id) AND (t2.feid = t3.feid) JOIN table_f AS t4 ON t3.exchange = t4.exchange) tU) limit 10; +---------+ | deal_id | +---------+ | 30 | | 82 | | 83 | | 108 | | 155 | | 156 | | 162 | | 184 | | 186 | | 195 | +---------+ 10 rows in set (10.62 sec)

1 0

[Maria-developers] my_apc.cc and new innodb
by Sergei Golubchik 19 Oct '12

19 Oct '12

Hi. There's some issue we need to discuss. I've discovered it when merging your APC code into 10.0-monty, but the problem is simple enough and could've been found earlier too. In the thd_killed() you do: 3809 extern "C" int thd_killed(const MYSQL_THD thd) 3810 { 3811 if (!thd) 3812 thd= current_thd; 3813 3814 Apc_target *apc_target= (Apc_target*)&thd->apc_target; 3815 if (apc_target->have_apc_requests()) 3816 apc_target->process_apc_requests(); 3817 3818 if (!(thd->killed & KILL_HARD_BIT)) 3819 return 0; 3820 return thd->killed; 3821 } But there's no rule that thd_killed() cannot be called with thd different from current_thd. That is, with thd of *another thread*. Of course, when it happens, the caller is responsible for making sure that the owner thread of that thd isn't going to use it concurrently. And that's exactly what happens in the new InnoDB. Now, show explain wasn't designed to work in this situation, it created items (which are created using sql_alloc, that uses mem_root of the current thread, and added to the free item list of the current thd). As a short-term fix, I've added a protection to thd_killed() to not call apc functions if thd != current_thd. But I don't like it, I think a better solution would be to switch thds, like thread pool does, and process apc requests anyway. What do you think? Regards, Sergei

1 0

[Maria-developers] libmysqlclient16 for MariaDB 5.5?
by Jim Nelin 19 Oct '12

19 Oct '12

Hi guys! I just recently joined this team/list, so I'm going to be polite and introduce myself. My name is Jim, work mainly as a server technician and developer and I've been using MariaDB for the past 2 years or so. Just recently i upgraded one of my older MySQL 5.1 nodes to MariaDB 5.5 - which comes with a lot of nice new features (I've previously only used MariaDB 5.1 and 5.2) I mainly use Debian nodes (stable/squeeze) for my and my clients projects. One thing i discovered shortly after the first upgrade was that there was no libmysqlclient16 (or libmariadbclient16) for the 5.5 release of MariaDB - only the new libmysqlclient18. This causes my applications to still use the older MySQL 5.1 release of libmysqlclient16 from the stable debian repo. As a lot of packages in Debian Squeeze (and squeeze-backports) still depends on libmysqlclient16 (php5-mysql, postfix-mysql, dovecot-common to mention a few.) It still works without any problems, with the stable/squeeze 5.1 mysqlclientlib16 (afaik anyway), but seems a bit risky to use an older client-lib against a newer server (I'm not aware of the changes in the client lib between 5.1 and 5.5 tho). The work-around this (to make it "a bit" better) was to manually install the .deb packages from MariaDB 5.3 which DO contain libmysqlclient16 (libmysqlclient16-5.3.9-mariadb117~squeeze). This still isn't "perfect" in my opinion - i would love to have a real lib(mysql|mariadb)client16 for the 5.5 release on Debian Squeeze. This is my current "setup" is the following: http://paste.debian.net/hidden/7c8219e0/ (basicly the client16-lib from 5.3 against a 5.5 server). I understand that it won't be any performance increases or similar due to older applications not using the new features in libmysqlclient18 or mDB 5.5. But it would help me to get rid of the warning below in phpMyAdmin and also make me feel more safe with the setup - a more consistent setup. "Your PHP MySQL library version 5.3.9-MariaDB differs from your MySQL server version 5.5.27. This may cause unpredictable behavior." So, the questions is - why is there isn't any libmysqlclient16 for MariaDB 5.5? And it possible to add it to the repos? If not - why? Can this be solved with a virtual package or wrapper-lib of some kind against libmariadbclient18? Please correct me if I've got something wrong. Thanks in advance, Best Regards -- Jim Nelin | +46 (0)720 454 909 Jine | http://jine.se | jim(a)jine.se

2 1

[Maria-developers] Fwd: XA problems in mysql-5.6.7-rc
by Rich Prohaska 19 Oct '12

19 Oct '12

Hello, I have been debugging a deadlock problem in the 2PC logging code in MySQL 5.6.7-rc. Luckily, Sergei pointed me to a bug fix. I integrated this fix into MySQL 5.6.7, and with a couple of tweaks, the deadlock problem is fixed. I verified the fix with helgrind, since helgrind reported a lock ordering violation in the original 5.6.7 code. My tweaks are: 1. Move the mutex unlock to the right place in the TC_LOG_MMAP::log_xid function. WRT the main maria trunk, move line 7087 of log.c to after the break, since the code expects the page lock to be held after the for loop. 2. helgrind reported a race on tc_log_cur_pages_used, so I used statistic_increment on line 6983. Also need to build with SAFE_STATISTICS so that the statistics counters are atomically modified. ---------- Forwarded message ---------- From: Sergei Golubchik <serg(a)askmonty.org> Date: Thu, Oct 11, 2012 at 4:04 PM Subject: Re: XA problems in mysql-5.6.7-rc To: Rich Prohaska <prohaska(a)tokutek.com> Cc: internals(a)lists.mysql.com Hi, Rich! On Oct 11, Rich Prohaska wrote: > Hello Sergei, > Can you give me a pointer to a source files, or a source control changeset? This is the bug report: https://bugs.launchpad.net/maria/+bug/578117 And this is the patch: http://bazaar.launchpad.net/~maria-captains/maria/5.1/revision/2881 > On Thu, Oct 11, 2012 at 3:54 PM, Sergei Golubchik <serg(a)askmonty.org> wrote: > > On Oct 11, Rich Prohaska wrote: > >> Hello All, > >> > >> TokuDB supports XA and runs in both MariaDB and MySQL. It works in > >> MariaDB without any patches. Unfortunately, there are several XA bugs > >> related to multiple XA storage engines in MySQL 5.6.7. It would be > >> great if these bugs were fixed in MySQL. One way to test these > >> changes is to include another example storage engine that supports XA. > >> This storage engine does not even need to work; it just needs to > >> export the prepare method in the handlerton interface. > >> > >> There is a block of code in handler.cc surrounded by > >> WILL_BE_DELETED_LATER. If I disable the debug assert, what else do I > >> need to worry about? > > > > There is a possible deadlock in the MySQL XA code. I don't know how > > probable it is to get it in practice, we've found it with our mutex > > deadlock detector (MariaDB only). I had to rework XA code in MariaDB > > quite a bit to get rid of the deadlock. > > Regards, Sergei

1 0

[Maria-developers] MariaDB and spatial indexes
by Vangelis Katsikaros 16 Oct '12

16 Oct '12

Hi I completed my MSc thesis, that was performed with the co-supervision of Sergei Golubchik from Monty Program AB. It's titled "Towards the universal spatial data model based indexing and its implementation in MySQL" which translates into doing a literature overview for R-tree based spatial indexes, GiST trees (Generalized Search Trees) and implementing them in MyISAM. The work is in progress! In case you are interested in the algorithm details of these indexes: * Thesis text: http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6394 * Current patches: https://www.dropbox.com/s/bw2cq2525zypi6m/patches.tar.gz Many many thanks to Sergei for all his help and support :) Regards Vangelis

1 0

[Maria-developers] FYI: SHOW EXPLAIN pushed to 10.0-base
by Sergei Petrunia 06 Oct '12

06 Oct '12

Hello, FYI, SHOW EXPLAIN has been merged and pushed to 10.0-base. It was not pushed into 5.5. Just in case, the final 5.5-based tree with SHOW EXPLAIN patch can be found here: https://code.launchpad.net/~maria-captains/maria/5.5-show-explain BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] Global transaction ID
by Kristian Nielsen 05 Oct '12

05 Oct '12

Hi Kristian, Monty asked me to connect with you regarding global transaction IDs in MariaDB. I reviewed some time ago the MySQL 5.6 global transaction ID design and implementation. Unfortunately I really did not like what I found, to the point that I decided this was not something we would want to have or support in MariaDB (see https://lists.launchpad.net/maria-developers/msg04837.html for an ealier mail on this). Both you and Giuseppe also had critisisms about it. Instead I came up with a (in my opinion) better design. I have now written this up in (great) detail here: https://mariadb.atlassian.net/browse/MDEV-26 It would be great if you (/your team) could comment on this and how it relates to what you need or any comments/suggestions etc. I know it is a long text, and a complex problem, so also feel free to just ask me for any clarifications or explanations you would like, even without digging through the whole text. The basic idea is to keep the existing concept of a stream of replication events and a slave's current position within this stream, but in a way that is uniform across the whole multi-level replication topology (global transaction ID), and that facilitates simple scripting of automated promotion of a new master. I have thought for a long time over this problem and how to best approach it and make it work well both with existing replication usage and with new features like multi-source and parallel replication. It is hard to get right, but I am fairly pleased with what I have ended up with so far. I am hopeful that this will be a good way forward, but I very much welcome any comments or alternate suggestions. Regards, - (the other) Kristian.

2 2

[Maria-developers] MariaDB Optimization for MySQL Users
by Daniel Bartholomew 04 Oct '12

04 Oct '12

MariaDB contains many new options and optimizations which, for compatibility or other reasons, are not enabled in the default install. Enabling them helps you gain extra performance from the same hardware when upgrading from MySQL to MariaDB. Monty thought this would be a good article for the Knowledgebase and I agree. I've created a stub article for this using a few short items Monty provided (just to get the ball rolling), and I am now soliciting suggestions from any and all of you so I can expand it. https://kb.askmonty.org/en/mariadb-optimization-for-mysql-users/ Please send me your MariaDB-only tips and recommended settings and other things people should enable or tweak after upgrading to MariaDB and I'll add them to the page (or feel free to do it yourself). Thanks! -- Daniel Bartholomew Google+ - http://gplus.to/dbart Twitter - http://twitter.com/daniel_bart MariaDB: An Enhanced Drop-in Replacement for MySQL Website - http://mariadb.org Twitter - http://twitter.com/mariadb Google+ - http://google.com/+mariadb Facebook - http://fb.com/MariaDB.dbms Knowledgebase - http://kb.askmonty.org Monty Program - http://montyprogram.com

1 0