Re: [Maria-developers] [Maria-discuss] Known limitation with TokuDB in Read Free Replication & parallel replication ?
Rich Prohaska
I rearranged the tokudb lock request wait function a little bit, and got the lock tree unit tests to compile again (since the API changed). commit on this tree: https://github.com/prohaska7/mariadb-server/tree/toku_opr3
Thanks! So I took a look at the complete patch we have so far, thinking what remains to do to get this included in main MariaDB. Do you have any suggestions for how/whether to upstream the TokuDB parts? I assume the new kill_query functionality would make sense to upstream. The callback to report waits could be upstreamed, but would not be used in non-mariadb builds, probably. And the actual tokudb_lock_wait_needed_callback() would probably not be upstreamed (or would go in #ifdef MARIADB_BASE_VERSION perhaps). Any thoughts? I still need to look into the should_retry_lock_requests, currently I get hangs if it is enabled. And I want to see if the wait_needed_callback interface can be cleaned up a bit. Otherwise the patch looks fairly complete to me, do you agree? I think this could go in 10.1, to fix optimistic parallel replication with TokuDB (which does not currently work at all). - Kristian.
Hello Kristian,
I am working on a replacement algorithm for the retry all lock_requests
function. Still in development. The race is that some locks are released
while some other thread is running the lock retry code. These locks are
not retried, so a blocked lock request is not handed the lock and it times
out.
Back in the day, we were thinking of attaching the pending lock requests to
the conflicting locks so that when the conflicting locks are released, it
is easy to find the pending lock requests. This change is beyond the scope
of my current work.
I have the PerconaFT changes to support MariaDB's optimistic parallel
replication on a branch that is suitable for merging into Pecona's repo:
https://github.com/prohaska7/tokuft/tree/killwait
Need to run some replication benchmarks that compare conservative vs
optimistic replication performance. This might lead to some interesting
results.
On Tue, Aug 30, 2016 at 10:24 AM, Kristian Nielsen wrote: Rich Prohaska I rearranged the tokudb lock request wait function a little bit, and got
the lock tree unit tests to compile again (since the API changed).
commit
on this tree: https://github.com/prohaska7/
mariadb-server/tree/toku_opr3 Thanks! So I took a look at the complete patch we have so far, thinking what
remains
to do to get this included in main MariaDB. Do you have any suggestions for how/whether to upstream the TokuDB parts? I
assume the new kill_query functionality would make sense to upstream. The
callback to report waits could be upstreamed, but would not be used in
non-mariadb builds, probably. And the actual
tokudb_lock_wait_needed_callback() would probably not be upstreamed (or
would go in #ifdef MARIADB_BASE_VERSION perhaps). Any thoughts? I still need to look into the should_retry_lock_requests, currently I get
hangs if it is enabled. And I want to see if the wait_needed_callback
interface can be cleaned up a bit. Otherwise the patch looks fairly
complete
to me, do you agree? I think this could go in 10.1, to fix optimistic parallel replication with
TokuDB (which does not currently work at all). - Kristian.
participants (2)
-
Kristian Nielsen -
Rich Prohaska