Hi, Kristian! On Jul 03, Kristian Nielsen wrote:
Sergei Golubchik <serg@mariadb.org> writes:
I'm kind of lost here. I really don't like requiring engines to do more obligatory things, but I don't see a good way around it either.
Agree, that's how I felt while trying to think of _any_ way to solve these problems. I ended up with current solution, but not entirely happy about it.
At least, we have some more ideas now to improve things.
What should be the next step?
As you suggested on irc, it would make sense to make a smaller innodb/xtradb only fix in 10.0 and a more engine-friendly, with the new api, in 10.1
I thought that if a transaction is rolled back, you can, technically force *all* later transactions to roll back too and redo them all. But that'd be kind of expensive.
It's not the expensive that worries me. The problem is that some of the following transactions may not be possible to roll back.
Ah, yes, indeed. We could still 1. rollback regardless and possibly break replication in this case. saying that a transactional engine will work without modifications in most cases, but not when it's mixed with non-trans updates 2. as discussed, have a flag to mark non-trans-updates transactions and don't run them in parallel at all. then a transactional engine will work without modifications. but that's for 10.1, if we do innodb-only fix in 10.0, it means we aren't concerned with other engines there.
But in some cases the deadlock, kill, and retry is unnecessary. This is about the issue in MDEV-5914, for example:
T1: UPDATE t1 SET secondary=NULL WHERE primary=1 T2: DELETE t1 WHERE secondary <= 3
If T1 and T2 were group-committed together on the master, we know that it is safe to run them in parallel, so we can relax the gap lock taken by T2 that can block T1. Then we avoid having to rollback and retry T2.
But if T2 committed in a different group commit on the master, we do not know that this relaxing of gap locks is safe. Because T2 may still run in parallel with the COMMIT of T1, we need to keep the gap locks, so that T2 will wait if it needs to access any row modified by T1.
How can T2 run in parallel with T1 if they're from different groups?
The reason I think my original API is simpler is that there is just one very simple thing that the engine must do: call thd_report_wait_for(T1,T2) whenever T1 is about to wait until T2 commits. Everything else is optional optimisations that can be safely ignored. It seems that any conceivable engine must already have easy access to knowledge about one transaction needing to wait for another. And indeed, in InnoDB only a couple of lines need to be added to handle this. So it is very simple to use. With your suggestion, however, things are more complex for the engine. It needs to implement additional deadlock detection, based on some enforced commit order. The very concept of "enforced commit order" is probably something new. And it needs to implement some way to resolve deadlocks with another transaction T2 that may or may not be running inside the engine. So again, a new concept is introduced, "Is transaction T running inside this engine"?
Hmm, okay... When you put it this way, it does sound simpler. Allright, let's keep thd_report_wait_for() :) Regards, Sergei