Thanks Kristian. That's a nice optimization that is difficult to do outside the DBMS engine.  

Cheers, Robert


On Wed, Sep 10, 2014 at 2:39 PM, Kristian Nielsen <knielsen@knielsen-hq.org> wrote:
Robert Hodges <robert.hodges@continuent.com> writes:

> There is one thing I have never understood about your parallel apply
> algorithm.  How do you handle the case where the server crashes when some
> threads have committed but others have not?  It seems as if you could have
> a problem with recovery.

Transactions are executed in parallel, but they are committed sequentially.

So if we crash, there is always a single point before which all transactions
are committed and after which no transactions are.

If using InnoDB and global transaction ID, this point is saved in a crash-safe
way, so no problems with crash recovery.

If using MyISAM, or if using old filename/offset position in relay-log.info,
then a crash may leave an inconsistency - but this is also the case with
non-parallel replication.

A substantial effort was spent in the code to make the sequential commit of
the parallelly applied transaction efficient. For example, it is possible for
such transactions to be group committed.

 - Kristian.