Re: [Maria-developers] safe patches to backport -- pool-of-threads
Hi! Sorry for replying this 'off-thread', but I was not by some strange reason subscribed to the maria-developers list :( MARK CALLAGHAN writes: mark> One of the features listed here mark> http://askmonty.org/wiki/index.php/MariaDB_versus_MySQL as a potential mark> backport is support for pool-of-threads. I am not sure how safe this mark> feature is today. mark> mark> http://mysqlha.blogspot.com/2009/01/no-new-global-mutexes-and-how-to-make.ht... mark> describes some of the problems that might cause a server to hang when mark> pool-of-threads is used. The main port of the pool-of-threads code has been in extensive use by a high performance MySQL customer for a long time and, within their limits, it has given them a lot of extra performance. To be fair, the above customer is using a scheduler that is using getport instead of libevent. I am considering to add also getport support to the scheduler.cc in MariaDB, even if this only gives higher performance on Solaris. The patch has it's problems and is not complete, but for certain workloads and configurations it can definitely save the day. One example is if your application needs 200,000 connected users. One of the biggest problem with pool-of-threads is that when you run out of threads you are stuck as you can't see what's going on and you can connect and kill things. The extra-port options fixes that. It also allows you to mix pool-of-threads and one-thread-per-connection in an easy way. It would have been better to make this a connection option, but because connections are also handled by the thread pool, when you run with pool-of-threads, this doesn't work as if all worker threads are occupied the connection will never go trough. Adding another pool for connection would require so much new work that it wouldn't be safe for MariaDB 5.1. For MariaDB 5.2 we should consider looking at adding another thread pool for connections and making the scheduler a connection option. Drizzle is by default using pool-of-threads, so the code gets more tested every day. mark> http://bugs.mysql.com/bug.php?id=42288 describes the performance mark> overhead of pool-of-threads. The overhead is significant for workloads mark> with fast queries such as sysbench. Brian/Drizzle is, as far as I know, trying to benchmark the original getevent port code and the libevent code to find out how things really stands. Brian suspect that there may a problem in libevent that needs to be fixed and he has someone looking into it. When they or we have fixed it, we will collaborate to ensure both projects works properly. mark> I think that pool-of-threads needs a lot of work before it can be mark> considered safe. We backported this to our 5.0 branch at work and have mark> begun to search for fixes. At least pool-of-threads is not default in MariaDB and you have to turn it explicitely one if you want to use it; This makes it reasonable safe to include into MariaDB 5.1 at this point as old user will not be affected at all by it and new users trying it can do that as long as they are aware of the limitations. Regards, Monty
participants (1)
-
Michael Widenius