Re: [Maria-developers] 1 Transaction and several threads
Hi, If we talk about bulk load I would write directly .ibd file format pages, now as InnoDB has always some order between pages doing this in parallel has challenges, we can read and parse input, sort it, reformat to InnoDB pages, write these pages (maybe you could do this also parallel). To speed this up, I would remove secondary keys that are not needed from table and rebuild them after bulk load. R: Jan On Thu, Aug 3, 2017 at 12:09 PM, Oleksandr Byelkin <sanja@mariadb.com> wrote:
Hi!
I am thinking about efficient inserting in a table using several threads (LOAD), is it possible (and how difficult if it does) to make inserts with help of several threads abut inside one InnoDB transaction (maybe one XA transaction)?
Hi, Sanja! FYI: https://jira.mariadb.org/browse/MDEV-515 and https://jira.mariadb.org/browse/MDEV-5835 On Aug 03, Jan Lindström wrote:
Hi,
If we talk about bulk load I would write directly .ibd file format pages, now as InnoDB has always some order between pages doing this in parallel has challenges, we can read and parse input, sort it, reformat to InnoDB pages, write these pages (maybe you could do this also parallel). To speed this up, I would remove secondary keys that are not needed from table and rebuild them after bulk load.
R: Jan
On Thu, Aug 3, 2017 at 12:09 PM, Oleksandr Byelkin <sanja@mariadb.com> wrote:
I am thinking about efficient inserting in a table using several threads (LOAD), is it possible (and how difficult if it does) to make inserts with help of several threads abut inside one InnoDB transaction (maybe one XA transaction)?
Regards, Sergei Chief Architect MariaDB and security@mariadb.org
Hi! Am 03.08.2017 um 13:39 schrieb Sergei Golubchik:
Hi, Sanja!
FYI:
https://jira.mariadb.org/browse/MDEV-515 and https://jira.mariadb.org/browse/MDEV-5835
It is https://jira.mariadb.org/browse/MDEV-13048
On Aug 03, Jan Lindström wrote:
Hi,
If we talk about bulk load I would write directly .ibd file format pages, now as InnoDB has always some order between pages doing this in parallel has challenges, we can read and parse input, sort it, reformat to InnoDB pages, write these pages (maybe you could do this also parallel). To speed this up, I would remove secondary keys that are not needed from table and rebuild them after bulk load.
R: Jan
On Thu, Aug 3, 2017 at 12:09 PM, Oleksandr Byelkin <sanja@mariadb.com> wrote:
I am thinking about efficient inserting in a table using several threads (LOAD), is it possible (and how difficult if it does) to make inserts with help of several threads abut inside one InnoDB transaction (maybe one XA transaction)? Regards, Sergei Chief Architect MariaDB and security@mariadb.org
_______________________________________________ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
participants (3)
-
Jan Lindström -
Oleksandr Byelkin -
Sergei Golubchik