----- Original Message -----
Pavel Ivanov <pivanof@google.com> writes:
So the slave coordinator (or I don't remember how you call it) reads relay log ahead of the last executing transaction? I.e. it will read and assign to threads T1.1, T1.2, then it will read T1.3, detect that there are no threads available for execution, but according to what you said it will still put this in the queue for thread 1, right? How long this queuing can be?
Does it keep all queued events in memory?
Does it depend on the size of the transactions (i.e. how much memory can it consume by this queuing)?
Right. The queueing is limited by the configuration variable --slave-parallel-max-queued, which defaults to 128KB per worker thread. It does not depend on the size of the transactions (it is possible to replicate a large transaction without keeping all of it in memory at once). It does need to keep at least one event in-memory per worker thread, of course, even if an individual event exceeds --slave-parallel-max-queued.
https://mariadb.atlassian.net/browse/MDEV-7202 for patch to add a status variable ....
Then I'd suggest to not add any special processing of such use case, but add something that will allow to easily monitor what happens. E.g. some status variables which could be plotted over time and show (or at least hint on) whether this is significant bottleneck for performance or not. This could be something like total time (in both wall time and accumulated CPU time) spent executing transactions in parallel, time spent rolling back transactions due to this lock conflict, time spent rolling back transactions because of other reasons (e.g. due to STOP SLAVE or reconnect after master crash), maybe also time spent waiting in one parallel thread while transaction is executing in another thread, etc.
added https://mariadb.atlassian.net/browse/MDEV-7340 quoting this
Yes, I agree, we need more of this. I think the monitoring part of the feature is currently rather weak, it probably suffers from it being now a long time since I was doing operations. Hopefully this can be significantly improved in the near future.
I wonder if such accumulated-time measurements can be added liberally without significantly affecting performance?
If each thread accumulate its own and if there needs to be a global it can be done like the global collation in the MDEV-7202 patch. -- -- Daniel Black, Engineer @ Open Query (http://openquery.com.au) Remote expertise & maintenance for MySQL/MariaDB server environments.