[Maria-developers] Database corruption
Hello, We have a java tools to massively load data in a database (parallel load in different tables). With it, on bb-10.2-compatibility branch, I'm able to reproduce crash and sometimes database corruption: · on truncate (but delete works fine) · on parallel insert Crash seems occurred in innoDB code. See attached, logs of each case. Do you need more information ? Regards, Jérôme.
I wonder if this could be caused by missing MDL protection around TRUNCATE TABLE, which would then incorrectly allow concurrent INSERT and TRUNCATE. One place where MDL is definitely missing is DROP DATABASE (reported as MDEV-11342 <https://jira.mariadb.org/browse/MDEV-11342>). That said, the InnoDB in MariaDB 10.2 is based on MySQL 5.7, and one difference between MySQL 5.6 and 5.7 is that InnoDB supports crash-safe atomic TRUNCATE TABLE for .ibd files (WL#6501 <http://dev.mysql.com/worklog/task/?id=6501>). Best regards, Marko -- DON’T MISS M|17 April 11 - 12, 2017 The Conrad Hotel New York City https://m17.mariadb.com/ Marko Mäkelä, Lead Developer InnoDB MariaDB Corporation
Hello Marko, Jerome, Marko, thanks for your comments. On 04/03/2017 08:17 AM, Marko Mäkelä wrote:
This is assigned to Sergey Vojtovich. Sergey is currently on vacation and will be back in the last decade of April. Marko, do you think it is something hard to fix? (Sorry, I'm not strong in this area of the code)
Hello Marko, Alexander, Thanks for your quick analysis. As I be able to reproduce crash with parallel "delete / insert", does it mean that "delete" act as truncate when there is no clause ? Feel free to ask me if you want me to test patches. Alexander, there is nothing specific to sql_mode=ORACLE. Best regards, Jérôme.
Hi Jérôme, Because InnoDB is a transactional storage engine when it comes to DML operations, DELETE FROM table (without a WHERE clause) is not equivalent to TRUNCATE TABLE. The reason is that you can ROLLBACK after DELETE, but you cannot ROLLBACK after TRUNCATE, which is a DDL operation (and DDL is not fully transactional). That said, I now have a better explanation for the problems that you have encountered. The branch bb-10.2-compatibility recently merged the branch 10.2 (which is close to what will be released as MariaDB Server 10.2.5). As part of this merge, a regression that was introduced in 10.1 was merged. That regression would cause a SIGSEGV during TRUNCATE TABLE (and presumably less likely during other DDL operations) when certain pages of the table are not in the InnoDB buffer pool. I pushed the fix (MDEV-12428 <https://jira.mariadb.org/browse/MDEV-12428>) to 10.1 and merged to 10.2 today. A merge of that to bb-10.2-compatibility should fix the issue. Best regards, Marko
Hi Bar, On Tue, Apr 4, 2017 at 9:16 AM, Alexander Barkov <bar@mariadb.org> wrote:
Please let me know when the fix is available in 10.2. I will propagate it to bb-10.2-compatibility asap.
I merged it yesterday to 10.2. Both bb-10.2-compatibility and 10.3 will need the merge of https://github.com/MariaDB/server/commit/6d417a0bad205a6bacfee10dbc46dd631b0... or a later revision of 10.2. Marko -- DON’T MISS M|17 April 11 - 12, 2017 The Conrad Hotel New York City https://m17.mariadb.com/ Marko Mäkelä, Lead Developer InnoDB MariaDB Corporation
Hello Marko, Alexander, Thanks for your works. When this issue will be corrected on bb-10.2-compatibility, I replay my tests. Best regards, Jérôme. De : Marko Mäkelä [mailto:marko.makela@mariadb.com] Envoyé : mardi 4 avril 2017 08:14 À : jerome brauge Cc : Alexander Barkov; MariaDB Developers (maria-developers@lists.launchpad.net); Sergey Vojtovich Objet : Re: [Maria-developers] Database corruption Hi Jérôme, Because InnoDB is a transactional storage engine when it comes to DML operations, DELETE FROM table (without a WHERE clause) is not equivalent to TRUNCATE TABLE. The reason is that you can ROLLBACK after DELETE, but you cannot ROLLBACK after TRUNCATE, which is a DDL operation (and DDL is not fully transactional). That said, I now have a better explanation for the problems that you have encountered. The branch bb-10.2-compatibility recently merged the branch 10.2 (which is close to what will be released as MariaDB Server 10.2.5). As part of this merge, a regression that was introduced in 10.1 was merged. That regression would cause a SIGSEGV during TRUNCATE TABLE (and presumably less likely during other DDL operations) when certain pages of the table are not in the InnoDB buffer pool. I pushed the fix (MDEV-12428<https://jira.mariadb.org/browse/MDEV-12428>) to 10.1 and merged to 10.2 today. A merge of that to bb-10.2-compatibility should fix the issue. Best regards, Marko
participants (3)
-
Alexander Barkov -
jerome brauge -
Marko Mäkelä