[Maria-developers] Some notes on pool_of_threads
Hi there, I've been playing around with the pool_of_threads addition and here is some of what I've found: All of this was on Mac OSX 10.5 1) Biggie here: Running the server with the --extra-port=NNNN option (with pool-of-threads set) causes the server to start then immediately stop. I'm still looking through logs and trace files to see if I can find anything meaningful. I am able to run without setting extra-port though. 2) We can set extra-max-connections when --extra-port isn't set. The limit is 100000, at least on my system. As I haven't been able to get the server running with extra-port yet, I haven't tested much beyond this. 3) We can also set thread-pool-size even if thread-handling isn't set to pool-of-threads. It's limit appears to be < 2560 (I was running into some mess on my system so I haven't pinpointed it yet). The biggest thing is that if thread-pool-size is set too large, we just get a start and stop (Once again, my logs are a bit of a mess due to 'play', so I'll have more info on the server behavior in a bit). 4) I found that if the number of connections is = thread-pool-size, new connections will terminate within 'connect-timeout' seconds (there was some question about whether we had to wait for the denied connection to receive a message until after a thread was freed). 5) Due to these problems (most especially #1), main.pool_of_threads.test fails on my machine. If I remove the --extra-port stuff, the test will run more than otherwise (the server dies quickly with extra-port stuff). For extra-max-connections and thread-pool-size, do we want to allow these to be set even if they aren't relevant to the way the server is currently running? Is there any specific information I can provide if anyone wants to look into why the server dies when I try to use --extra-port? Testing notes: 1) I have some changes to make to pool_of_threads.test (style, additional testing, etc). 2) We should also add some testing for extra-max-connections, thread-pool-size, etc. Possibly to variables.test, though somewhere else would work. I've played around a bit with dynamically changing the variables, etc. I just can't get MTR to run the test to really polish my proposed changes before submitting a patch. Thanks, Patrick.
Hi!
"Patrick" == Patrick Crews <gleebix@gmail.com> writes:
Patrick> Hi there, Patrick> I've been playing around with the pool_of_threads addition and here is some Patrick> of what I've found: Patrick> All of this was on Mac OSX 10.5 Patrick> 1) Biggie here: Running the server with the --extra-port=NNNN option (with Patrick> pool-of-threads set) causes the server to start then immediately stop. I'm Patrick> still looking through logs and trace files to see if I can find anything Patrick> meaningful. I am able to run without setting extra-port though. Very strange. Works for me and this is also tested in t/pook_of_threads. Does the following work for you: cd mysql-test ./mysql-test-run t/pool_of_threads If yes, can can you start the server with cd mysqld ./mysqld --no-defaults --debug --extra-port=9999 and when it aborts, check the /tmp/mysqld.trace file for clues. (Search from the end after 'error') (For the above to work, you should configure MariaDB with --extra-debug). Patrick> 2) We can set extra-max-connections when --extra-port isn't set. The limit Patrick> is 100000, at least on my system. As I haven't been able to get the server Patrick> running with extra-port yet, I haven't tested much beyond this. You never need to set extra-max-connections to more than 2000. Why trying to set extra-max-connection to 10000 ? Normally you should set max-connections to a high number and extra-max-connections to 100 or similar. Patrick> 3) We can also set thread-pool-size even if thread-handling isn't set to Patrick> pool-of-threads. It's limit appears to be < 2560 (I was running into some Patrick> mess on my system so I haven't pinpointed it yet). The biggest thing is Patrick> that if thread-pool-size is set too large, we just get a start and stop Patrick> (Once again, my logs are a bit of a mess due to 'play', so I'll have more Patrick> info on the server behavior in a bit). thread-pool-size should normally not be higher than the number of core * 2 you have in your system. At startup mysqld creates 'pool-of-threads' number of threads to handle your queries. Because of this, it's quite expected that you can't create more than 2580 threads; You will run out of virtual memory. This shouldn't however be a problem, as when you use pool-of-threads, you are not likely to want more than 8-16 threads. Patrick> 4) I found that if the number of connections is = thread-pool-size, new Patrick> connections will terminate within 'connect-timeout' seconds (there was some Patrick> question about whether we had to wait for the denied connection to receive a Patrick> message until after a thread was freed). When you use pool of threads (mysqld --thread-handling=pool-of-threads) there shouldn't normally be any timeouts for new connections; The timeout only happens if all threads in the pool is doing some work. Patrick> 5) Due to these problems (most especially #1), main.pool_of_threads.test Patrick> fails on my machine. If I remove the --extra-port stuff, the test will run Patrick> more than otherwise (the server dies quickly with extra-port stuff). It's very strange. It would be of great help if you could make a dbug trace for this. Patrick> For extra-max-connections and thread-pool-size, do we want to allow these to Patrick> be set even if they aren't relevant to the way the server is currently Patrick> running? Yes. The idea is that you can specify defaults in your .my.cnf file and then just start mysqld with different --thread-handling options without having to change anything else. (This is the same way all other MySQL options works). Patrick> Is there any specific information I can provide if anyone wants to look into Patrick> why the server dies when I try to use --extra-port? See above. A trace file would be very useful! Patrick> Testing notes: Patrick> 1) I have some changes to make to pool_of_threads.test (style, additional Patrick> testing, etc). Please propose! Patrick> 2) We should also add some testing for extra-max-connections, Patrick> thread-pool-size, etc. We have tests for extra-max-connections in connect.test and pool_of_threads.test thread-pool-size is a bit hard to test, but don't know if this needs to be explicitely tested as benchmarks will show how this works better than anything else. Patrick> Possibly to variables.test, though somewhere else Patrick> would work. I've played around a bit with dynamically changing the Patrick> variables, etc. I just can't get MTR to run the test to really polish my Patrick> proposed changes before submitting a patch. Anything we can help you with? Regards, Monty
participants (2)
-
Michael Widenius -
Patrick Crews