Hi, I implemented the last part of in-order commit, which pushes the wait into the transaction coordinator, so that group commit can work and performance can be good. On the one hand I am really pleased to get this done, it is something I have been thinking on for 3 years now. On the other hand I realise this is fairly complex stuff, so please be aware that I am 100% open to suggestions for any changes to this or other ideas on how to proceed. I ran some quick benchmarks. What I did was setup a master with 20000 independent inserts into a table. I run with sync_binlog=1, innodb_flush_log_at_trx_commit=1, and --log-slave-updates. The base MariaDB needs 71 seconds to replicate the 20000 transactions. Your original patch needs 12 seconds. With my in-order patch 15 seconds are needed. But with my in-order patch and increasing the number of threads from 16 to 24, then just 11 seconds are needed. 71 seconds Base 12 seconds Original @ 16 threads 15 seconds In-order @ 16 threads 11 seconds In-order @ 24 threads So for this quick benchmark, in-order is somewhat slower, but one can compensate for this by increasing the number of threads. This makes sense; with in-order there will be some threads waiting, so adding more threads is needed to ensure enough non-waiting threads to get full performance. This is great results I think, in-order appears quite viable performance-wise. There are a number of things that becomes easier when commits on the slave are guaranteed to be in-order (such as global transaction id). (And btw, that's 6 times faster replication without the user having to do anything special, which is also *very* nice! I am really looking forward to getting this fully integrated in MariaDB). On the other hand, it is clear that some workloads will suffer under in-order. For example something like this: UPDATE t1 SET a=5 WHERE id=10; UPDATE t1 SET a=4 WHERE id=10; UPDATE t1 SET a=3 WHERE id=10; UPDATE t1 SET a=2 WHERE id=10; UPDATE t1 SET a=1 WHERE id=10; UPDATE t1 SET a=1 WHERE id=20; UPDATE t1 SET a=2 WHERE id=20; UPDATE t1 SET a=3 WHERE id=20; UPDATE t1 SET a=4 WHERE id=20; UPDATE t1 SET a=5 WHERE id=20; With out-of-order, all the id=20 updates can run in parallel with all the id=10 updates. But with in-order, the first id=20 update will only commit after all the id=10 updates have run, so the remaining id=20 updates can not run in parallel. Performance will be slower, unless there are more events deeper down in the binlog which can be run in parallel instead. It is hard to predict how common such cases will be, but I am at least hopeful! To fix a potential deadlock with MyISAM, I changed conflict detection. Now for non-transactional tables, the hash key will have only the table name (not the PK values). Thus, any two updates of the same MyISAM tables will be a conflict. Updates to different tables are ok to run in parallel. I pushed the changes as usual to lp:~knielsen/maria/dingqi-parallel-replication/ and I attached the full patch. This completes the in-order experiment for me. Of course the patch still needs more work to be finished, like it should probably be possible to enable/disable in-order with an option etc. But probably first we should discuss more if in-order is a good idea at all, and in general how to proceed with the integration of the parallel replication feature. A big benefit of the in-order method is that users will be able to enable it without fear that their applications will break. Unlike for example the MySQL 5.6 multi-threaded slave, there is no need to partition the data into different schemas and audit/rewrite all applications to ensure no cross-schema queries. With in-order things will work exactly as normal, it is invisible to applications. Only thing is that row-based is required to get speedup, but everything works correctly even if some statement-based events turn up (and if we combine it with http://askmonty.org/worklog/Server-RawIdeaBin/?tid=184 then we can even do some statement-based parallel replication also using the in-order stuff). And we can still have out-of-order as an option. - Kristian.