- developers - lists.mariadb.org

[Maria-developers] Request for testing: MWL#182 SHOW EXPLAIN
by Sergei Petrunia 11 May '12

11 May '12

Hi Elena, I need testing for MWL#182 SHOW EXPLAIN. The code is here: lp:~maria-captains/maria/5.3-show-explain2 (note the '2' at the end) The branch is currently based on 5.3, although will move to 5.5 soon (I suppose this doesn't cause any problems for testing). Currently, I can think of two things that need testing: === "Asynchronous Procedure Call" module === This is a module that allows one thread to schedule a call in the other thread, and wait until the call has been invoked. Suggested way to test: - run a set of queries that look at each other with "SHOW EXPLAIN FOR <thread>" statements. - have some "victim" threads which run long queries, and have "attacker" threads which submit a lot of "SHOW EXPLAIN FOR <victim_thread>" queries. === On-the-fly EXPLAIN printing functionality === Before this WL, the code that produced EXPLAIN output - Required that optimizer produces a slightly different structures when EXPLAIN is running - Did destructive modifications to the query plan (that is, it was not possible to call these functions two times to get two EXPLAIN outputs for the same planned query) Suggested way to test: - Run various complicated queries, with UNIONs, JOINs, etc. - While they are running, do SHOW EXPLAIN FOR command for them, expecting it to crash or produce nonsensical output at some point. If SHOW EXPLAIN crashes, the crash report will show the query it was trying to obtain the EXPLAIN for. My current suggestion is to use some "runner" threads to run the queries and use another set of threads to hit the "runner" threads with SHOW EXPLAIN request. When creating tests in mysql-test/t/show_explain.test, I made a framework that allows to stop query execution at some particular point and then wait until some number of SHOW EXPLAIN requests were arrived and served. This requires insertion of "probes" into the source code, though. Perhaps, we could use something like that for your testing: to have the query pause at some "interesting" points and wait for SHOW EXPLAIN requests to arrive and be served. I wasn't able to come up with any intelligent way to find interesting point though, so this is still just an idea. BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 1

Re: [Maria-developers] Added --continue-on-error to mysqltest and mysql-test-run
by Sergei Golubchik 07 May '12

07 May '12

Hi, Monty! On May 04, Michael Widenius wrote: > At lp:maria/5.5 > > ------------------------------------------------------------ > revno: 3393 > committer: Michael Widenius <monty(a)askmonty.org> > branch nick: maria-5.5 > message: > Added --continue-on-error to mysqltest and mysql-test-run > This will contune the test case even if there was an error. This > makes it easier to run a test that contains many sub tests against > one engine. Monty, why did you do your parallel implementation of this feature, instead of using the existing abort_on_error flag? Regards, Sergei

1 0

Re: [Maria-developers] MDEV-255: Compile handlersocket plugin in 5.5
by Sergei Golubchik 07 May '12

07 May '12

Hi, Vladislav! On May 05, Vladislav Vaintroub wrote: > ------------------------------------------------------------ > revno: 3398 > revision-id: wlad(a)montyprogram.com-20120505003610-rz89d70v8c6n5v0c > parent: wlad(a)montyprogram.com-20120504152240-ptie9ox0vjqqukd4 > committer: Vladislav Vaintroub <wlad(a)montyprogram.com> > branch nick: 5.5 > timestamp: Sat 2012-05-05 02:36:10 +0200 > message: > MDEV-255: Compile handlersocket plugin in 5.5 > === added file 'plugin/handler_socket/CMakeLists.txt' > --- a/plugin/handler_socket/CMakeLists.txt 1970-01-01 00:00:00 +0000 > +++ b/plugin/handler_socket/CMakeLists.txt 2012-05-05 00:36:10 +0000 > @@ -0,0 +1,41 @@ > + > +IF(WIN32 OR CMAKE_SYSTEM_NAME MATCHES "FreeBSD") > + # Handlersocket does not compile on Windows, compiles but does > + # not start on FreeBSD. Why does it not start of BSD? > + RETURN() > +ENDIF() > + > +#Remove -fno-implicit-templates from compiler flags(handlersocket would not work with it) > +IF(CMAKE_COMPILER_IS_GNUCXX) > + STRING(REPLACE "-fno-implicit-templates" "" CMAKE_CXX_FLAGS ${CMAKE_CXX_FLAGS}) Do we still compile with -fno-implicit-templates? I thought we don't use it anymore. We shouldn't. > +ENDIF() > + > === modified file 'plugin/handler_socket/handlersocket/mysql_incl.hpp' > --- a/plugin/handler_socket/handlersocket/mysql_incl.hpp 2011-06-07 11:19:49 +0000 > +++ b/plugin/handler_socket/handlersocket/mysql_incl.hpp 2012-05-05 00:36:10 +0000 > @@ -13,7 +13,10 @@ > #define HAVE_CONFIG_H > #endif > > +#ifndef MYSQL_DYNAMIC_PLUGIN > #define MYSQL_DYNAMIC_PLUGIN > +#endif I prefer not to have #ifdef, but instead write #define MYSQL_DYNAMIC_PLUGIN 1 you apparently got macro redefinition warning, right? Because there was -DMYSQL_DYNAMIC_PLUGIN on the gcc command line? It defines MYSQL_DYNAMIC_PLUGIN to be 1, and when it's redefined in the source to the empty string you get a warning. "Redefining" it to the same value of 1 is okay, there will be no warning for it. That's why all such symbols should be defined to 1, not to an empty string. > + > #define MYSQL_SERVER 1 > Regards, Sergei

2 2

[Maria-developers] KB Question: Ubuntu 10.04 LTS Server and MariaDB 5.5.23
by Daniel Bartholomew 07 May '12

07 May '12

A question has been posted to the Knowledgebase: http://kb.askmonty.org/en/ubuntu-1004-lts-server-and-mariadb-5523 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Ubuntu 10.04 LTS Server and MariaDB 5.5.23 After installing fresh mariadb, I'm having troubles using some software compiled with the mysql support. For ex. mysql2 ruby gem: Incorrect MySQL client library version! This gem was compiled for 5.5.23-MariaDB but the client library is 5.1.61 Quick investigation took me into this conclusions: mariadb-client-5.5 introduces new libmariadbclient18, but depends on libdbd-mysql-perl which depends on libmysqlclient16. So after install we have 2 libraries: libmysqlclient.so.16(from original mysql) and libmysqlclient.so.18(from mariadb). And default libmysqlclient.so is symlinked to libmysqlclient.so.16 Can this be solved in some way? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -- Daniel Bartholomew MariaDB - http://mariadb.org Monty Program - http://montyprogram.com AskMonty Knowledgebase - http://kb.askmonty.org

2 1

[Maria-developers] A question about JOIN re-execution by subquery code
by Sergei Petrunia 07 May '12

07 May '12

Hi Sanja, As far I understand, JOIN structure has two "modules" to support join re-execution: M1. save_join_tab() saves/restores the contents of JOIN::join_tab array, as if execution process changed JOIN_TAB structures M2. JOIN::tmp_join is used to save/restore the contents of JOIN structure itself. What puzzles me are two questions: Q1: Why do we have separate functions to M1 and M2, instead of one "save everything" call, and one "restore everything" call? Q2: Why does M1 copy the whole JOIN_TAB structures? I've made a patch which makes it to save/restore JOIN::join_tab pointer instead (attached), and it passed all tests in the main test suite. (I am asking all this in connection with bug MDEV-238 on MWL#182 code) BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] MariaDB 5.3.7 release prep
by Daniel Bartholomew 04 May '12

04 May '12

All, The packages and other files for MariaDB 5.3.7 have been uploaded to the primary mirror and we're waiting for the other mirrors to pick up the new files before announcing the release. I've created preliminary changelog and release notes pages for the release here: http://kb.askmonty.org/en/mariadb-537-release-notes http://kb.askmonty.org/en/mariadb-537-changelog From the changelog, it appears this is primarily a bug-fix release. If there are any particularly notable bugs or features of this release which I should make mention of in the release notes, let me know ASAP. Thanks. -- Daniel Bartholomew MariaDB - http://mariadb.org Monty Program - http://montyprogram.com AskMonty Knowledgebase - http://kb.askmonty.org

1 0

[Maria-developers] proposed patch for handler APIs for range query information
by Zardosht Kasheff 02 May '12

02 May '12

Hello all, After talking with Colin Charles last night, I have been encouraged to send more patches for MariaDB. Here is a hopefully simple one that benefits all storage engines. Attached is a proposed patch of new handler APIs. The purpose of this patch is to give storage engines more information about impending range queries. The idea is if MySQL is about to perform a range query from a to b, then the handler is informed that a range query is about to happen. It extends functionality like handler::prepare_index_scan to other scenarios. We use it to know how much data to prefetch and prelock before doing a range query. Any storage engine can use this as a heuristic for optimizing range query performance. I would love to hear feedback -Zardosht

2 2

[Maria-developers] KB Question: Aria features ?
by Daniel Bartholomew 30 Apr '12

30 Apr '12

A question has been posted to the Knowledgebase: http://kb.askmonty.org/en/aria-features - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Aria features ? When Aria Engine is a new generation MyISAM Engine , whether the new features: - parallel index creating? - merge table? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Thanks. -- Daniel Bartholomew MariaDB - http://mariadb.org Monty Program - http://montyprogram.com AskMonty Knowledgebase - http://kb.askmonty.org

1 0

[Maria-developers] KB question: parallel index creating
by Daniel Bartholomew 30 Apr '12

30 Apr '12

A question has been posted to the Knowledgebase: http://kb.askmonty.org/en/parallel-index-creating - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - parallel index creating? Hi all can we expect the creation of parallel index creating in MyISAM engine? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Thanks. -- Daniel Bartholomew MariaDB - http://mariadb.org Monty Program - http://montyprogram.com AskMonty Knowledgebase - http://kb.askmonty.org

1 0

Re: [Maria-developers] Request for testing: MWL#182 SHOW EXPLAIN
by Sergei Petrunia 27 Apr '12

27 Apr '12

On Fri, Apr 27, 2012 at 01:31:08PM +0400, Elena Stepanova wrote: > Hi Sergey, > > OK, I will start working on this. > > > Before this WL, the code that produced EXPLAIN output > > - Required that optimizer produces a slightly different structures > > when EXPLAIN is running > > - Did destructive modifications to the query plan > > Does this mean that the traditional EXPLAIN (preceding a statement or > subsequent to it) is also affected? > Yes. I've changed regular EXPLAIN code so that it can be ran multiple times. That way, traditional EXPLAIN is affected. There is one known difference, currently only documented in this e-mail (will forward it): Date: Sat, 27 Aug 2011 10:14:13 +0400 From: Sergey Petrunya <psergey(a)askmonty.org> To: maria-developers(a)lists.launchpad.net Subject: [Maria-developers] MWL#182: a problem select_type column, SIMPLE vs PRIMARY > By the way, s there any standard way of naming the new EXPLAIN, as > opposed to the old EXPLAIN? (e.g. Online EXPLAIN? Asynchronous? Running? > Parallel?) > This is just to simplify further talks. We call it "SHOW EXPLAIN", following the SQL syntax. Nobody has made an explicit decision on the "official" way to name it, but it looks like this name will stick. > Regards, > /E > > > On 4/27/2012 9:13 AM, Sergei Petrunia wrote: >> Hi Elena, >> >> I need testing for MWL#182 SHOW EXPLAIN. >> >> The code is here: lp:~maria-captains/maria/5.3-show-explain2 (note the '2' at >> the end) >> >> The branch is currently based on 5.3, although will move to 5.5 soon (I suppose >> this doesn't cause any problems for testing). >> >> Currently, I can think of two things that need testing: >> >> === "Asynchronous Procedure Call" module === >> This is a module that allows one thread to schedule a call in the other thread, >> and wait until the call has been invoked. >> >> Suggested way to test: >> - run a set of queries that look at each other with "SHOW EXPLAIN FOR<thread>" >> statements. >> - have some "victim" threads which run long queries, and have "attacker" >> threads which submit a lot of "SHOW EXPLAIN FOR<victim_thread>" queries. >> >> === On-the-fly EXPLAIN printing functionality === >> Before this WL, the code that produced EXPLAIN output >> - Required that optimizer produces a slightly different structures when EXPLAIN >> is running >> - Did destructive modifications to the query plan (that is, it was not possible >> to call these functions two times to get two EXPLAIN outputs for the same >> planned query) >> >> Suggested way to test: >> - Run various complicated queries, with UNIONs, JOINs, etc. >> - While they are running, do SHOW EXPLAIN FOR command for them, expecting >> it to crash or produce nonsensical output at some point. >> If SHOW EXPLAIN crashes, the crash report will show the query it was trying >> to obtain the EXPLAIN for. >> >> My current suggestion is to use some "runner" threads to run the queries and >> use another set of threads to hit the "runner" threads with SHOW EXPLAIN >> request. >> >> When creating tests in mysql-test/t/show_explain.test, I made a framework that >> allows to stop query execution at some particular point and then wait until >> some number of SHOW EXPLAIN requests were arrived and served. >> >> This requires insertion of "probes" into the source code, though. >> >> Perhaps, we could use something like that for your testing: to have the query >> pause at some "interesting" points and wait for SHOW EXPLAIN requests to arrive >> and be served. I wasn't able to come up with any intelligent way to find >> interesting point though, so this is still just an idea. >> >> BR >> Sergei -- BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0