November 2011 - developers - lists.mariadb.org

Re: [Maria-developers] [Commits] Rev 3020: MWL#192: Non-blocking client API for libmysqlclient. in http://bazaar.launchpad.net/~maria-captains/maria/5.2
by Michael Widenius 09 Jan '12

09 Jan '12

Hi! >>>>> "knielsen" == knielsen <knielsen(a)knielsen-hq.org> writes: knielsen> At http://bazaar.launchpad.net/~maria-captains/maria/5.2 knielsen> ------------------------------------------------------------ knielsen> revno: 3020 knielsen> revision-id: knielsen(a)knielsen-hq.org-20110920104925-huxke80yuq1zrvcx knielsen> parent: monty(a)askmonty.org-20110816160131-4kju8si1nqqnohxg knielsen> committer: knielsen(a)knielsen-hq.org knielsen> branch nick: tmp2 knielsen> timestamp: Tue 2011-09-20 12:49:25 +0200 knielsen> message: knielsen> MWL#192: Non-blocking client API for libmysqlclient. knielsen> All client functions that can block on I/O have alternate _start() and knielsen> _cont() versions that do not block but return control back to the knielsen> application, which can then issue I/O wait in its own fashion and later knielsen> call back into the library to continue the operation. knielsen> Works behind the scenes by spawning a co-routine/fiber to run the knielsen> blocking operation and suspend it while waiting for I/O. This knielsen> co-routine/fiber use is invisible to applications. knielsen> For i368/x86_64 on GCC, uses very fast assembler co-routine support. On knielsen> Windows uses native Win32 Fibers. Falls back to POSIX ucontext on other knielsen> platforms. Assembler routines for more platforms are relatively easy to knielsen> add by extending mysys/my_context.c, eg. similar to the Lua lcoco knielsen> library. knielsen> For testing, mysqltest and mysql_client_test are extended with the knielsen> option --non-blocking-api. This causes the programs to use the knielsen> non-blocking API for database access. mysql-test-run.pl has a similar knielsen> option --non-blocking-api that uses this, as well as additional knielsen> testcases. knielsen> An example program tests/async_queries.c is included that uses the new knielsen> non-blocking API with libevent to show how, in a single-threaded knielsen> program, to issue many queries in parallel against a database. <cut> +++ client/async_example.c 2011-09-20 10:49:25 +0000 @@ -0,0 +1,207 @@ +/* + Copyright 2011 Kristian Nielsen and Monty Program Ab. + + Experiments with non-blocking libmysql. + + This is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 2 of the License, or + (at your option) any later version. + + This is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this. If not, see <http://www.gnu.org/licenses/>. +*/ Please change license to LGPL for this and the other new files (so we can add them to our new LGPL client library) + + +#ifndef __WIN__ +#include <poll.h> +#else +#include <WinSock2.h> +#endif You don't need to incldue Winsock2.h; This is automaticly included by mysql.h. +#include <stdlib.h> +#include <stdio.h> +#include <mysql.h> <cut> === modified file 'client/mysqltest.cc' --- client/mysqltest.cc 2011-05-18 13:17:26 +0000 +++ client/mysqltest.cc 2011-09-20 10:49:25 +0000 @@ -60,6 +60,12 @@ #define SIGNAL_FMT "signal %d" #endif === modified file 'include/Makefile.am' --- include/Makefile.am 2011-05-03 16:10:10 +0000 +++ include/Makefile.am 2011-09-20 10:49:25 +0000 @@ -46,7 +46,7 @@ atomic/rwlock.h atomic/x86-gcc.h \ atomic/generic-msvc.h \ atomic/gcc_builtins.h my_libwrap.h my_stacktrace.h \ - wqueue.h waiting_threads.h + wqueue.h waiting_threads.h my_context.h EXTRA_DIST = mysql.h.pp mysql/plugin_auth.h.pp mysql/client_plugin.h.pp CMakeLists.txt === added file 'include/my_context.h' --- include/my_context.h 1970-01-01 00:00:00 +0000 +++ include/my_context.h 2011-09-20 10:49:25 +0000 @@ -0,0 +1,222 @@ +/* + Copyright 2011 Kristian Nielsen + + Experiments with non-blocking libmysql. + + This is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 2 of the License, or + (at your option) any later version. + + This is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this. If not, see <http://www.gnu.org/licenses/>. +*/ + +/* + Simple API for spawning a co-routine, to be used for async libmysqlclient. + + Idea is that by implementing this interface using whatever facilities are + available for given platform, we can use the same code for the generic + libmysqlclient-async code. + + (This particular implementation uses Posix ucontext swapcontext().) +*/ + +#ifdef __WIN__ +#define MY_CONTEXT_USE_WIN32_FIBERS 1 +#elif defined(__GNUC__) && __GNUC__ >= 3 && defined(__x86_64__) +#define MY_CONTEXT_USE_X86_64_GCC_ASM +#elif defined(__GNUC__) && __GNUC__ >= 3 && defined(__i386__) +#define MY_CONTEXT_USE_I386_GCC_ASM +#else +#define MY_CONTEXT_USE_UCONTEXT +#endif + +#ifdef MY_CONTEXT_USE_WIN32_FIBERS +struct my_context { + void (*user_func)(void *); + void *user_arg; + void *app_fiber; + void *lib_fiber; + int return_value; +#ifndef DBUG_OFF + void *dbug_state; +#endif +}; +#endif + + +#ifdef MY_CONTEXT_USE_UCONTEXT +#include <ucontext.h> + +struct my_context { + void (*user_func)(void *); + void *user_data; + void *stack; + size_t stack_size; + ucontext_t base_context; + ucontext_t spawned_context; + int active; +#ifdef HAVE_VALGRIND_VALGRIND_H + unsigned int valgrind_stack_id; +#endif +#ifndef DBUG_OFF + void *dbug_state; +#endif +}; +#endif + + +#ifdef MY_CONTEXT_USE_X86_64_GCC_ASM +#include <stdint.h> + +struct my_context { + uint64_t save[9]; + void *stack_top; + void *stack_bot; +#ifdef HAVE_VALGRIND_VALGRIND_H + unsigned int valgrind_stack_id; +#endif +#ifndef DBUG_OFF + void *dbug_state; +#endif +}; +#endif + + +#ifdef MY_CONTEXT_USE_I386_GCC_ASM +#include <stdint.h> + +struct my_context { + uint64_t save[7]; + void *stack_top; + void *stack_bot; +#ifdef HAVE_VALGRIND_VALGRIND_H + unsigned int valgrind_stack_id; +#endif +#ifndef DBUG_OFF + void *dbug_state; +#endif +}; +#endif + + +/* + Initialize an asynchroneous context object. + Returns 0 on success, non-zero on failure. +*/ +extern int my_context_init(struct my_context *c, size_t stack_size); + +/* Free an asynchroneous context object, deallocating any resources used. */ +extern void my_context_destroy(struct my_context *c); + +/* + Spawn an asynchroneous context. The context will run the supplied user + function, passing the supplied user data pointer. + + The context must have been initialised with my_context_init() prior to + this call. + + The user function may call my_context_yield(), which will cause this + function to return 1. Then later my_context_continue() may be called, which + will resume the asynchroneous context by returning from the previous + my_context_yield() call. + + When the user function returns, this function returns 0. + + In case of error, -1 is returned. +*/ +extern int my_context_spawn(struct my_context *c, void (*f)(void *), void *d); + +/* + Suspend an asynchroneous context started with my_context_spawn. + + When my_context_yield() is called, execution immediately returns from the + last my_context_spawn() or my_context_continue() call. Then when later + my_context_continue() is called, execution resumes by returning from this + my_context_yield() call. + + Returns 0 if ok, -1 in case of error. +*/ +extern int my_context_yield(struct my_context *c); + +/* + Resume an asynchroneous context. The context was spawned by + my_context_spawn(), and later suspended inside my_context_yield(). + + The asynchroneous context may be repeatedly suspended with + my_context_yield() and resumed with my_context_continue(). + + Each time it is suspended, this function returns 1. When the originally + spawned user function returns, this function returns 0. + + In case of error, -1 is returned. +*/ +extern int my_context_continue(struct my_context *c); + + +struct mysql_async_context { + /* + This is set to the value that should be returned from foo_start() or + foo_cont() when a call is suspended. + It is also set to the event(s) that triggered when a suspended call is + resumed, eg. whether we woke up due to connection completed or timeout + in mysql_real_connect_cont(). + */ + unsigned int ret_status; + /* + This is set to the result of the whole asynchronous operation when it + completes. It uses a union, as different calls have different return + types. + */ + union { + void *r_ptr; + const void *r_const_ptr; + int r_int; + my_bool r_my_bool; + } ret_result; + /* + The timeout value, for suspended calls that need to wake up on a timeout + (eg. mysql_real_connect_start(). + */ + unsigned int timeout_value; + /* + This flag is set when we are executing inside some asynchronous call + foo_start() or foo_cont(). It is used to decide whether to use the + synchronous or asynchronous version of calls that may block such as + recv(). + + Note that this flag is not set when a call is suspended, eg. after + returning from foo_start() and before re-entering foo_cont(). + */ + my_bool active; + /* + This flag is set when an asynchronous operation is in progress, but + suspended. Ie. it is set when foo_start() or foo_cont() returns because + the operation needs to block, suspending the operation. + + It is used to give an error (rather than crash) if the application + attempts to call some foo_cont() method when no suspended operation foo is + in progress. + */ + my_bool suspended; + /* + If non-NULL, this is a pointer to a callback hook that will be invoked with + the user data argument just before the context is suspended, and just after + it is resumed. + */ + void (*suspend_resume_hook)(my_bool suspend, void *user_data); + void *suspend_resume_hook_user_data; + /* + This is used to save the execution contexts so that we can suspend an + operation and switch back to the application context, to resume the + suspended context later when the application re-invokes us with + foo_cont(). + */ + struct my_context async_context; +}; === modified file 'include/my_dbug.h' --- include/my_dbug.h 2011-02-20 16:51:43 +0000 +++ include/my_dbug.h 2011-09-20 10:49:25 +0000 @@ -83,6 +83,8 @@ extern void _db_unlock_file_(void); extern FILE *_db_fp_(void); extern void _db_flush_(); +extern void dbug_swap_code_state(void **code_state_store); +extern void dbug_free_code_state(void **code_state_store); Please define a macros DBUG_FREE_CODE_STATE / DBUG_SWAP_CODE_STATE that are mapped to the aboe if DBUG is defined and to "do {} while (0)" otherwise. This allows you to replace: #ifndef DBUG_OFF dbug_free_code_state(&c->dbug_state); #endif with DBUG_FREE_CODE_STATE(&c->dbug_state); (It's always nice to not have a lot of #ifdef in the code). +++ include/mysql.h 2011-09-20 10:49:25 +0000 void *extension; } MYSQL_PARAMETERS; +/* + Flag bits, the asynchronous methods return a combination of these ORed + together to let the application know when to resume the suspended operation. +*/ +typedef enum { + MYSQL_WAIT_READ= 1, /* Wait for data to be available on socket to read */ + /* mysql_get_socket_fd() will return socket descriptor*/ + MYSQL_WAIT_WRITE= 2, /* Wait for socket to be ready to write data */ + MYSQL_WAIT_EXCEPT= 4, /* Wait for select() to mark exception on socket */ + MYSQL_WAIT_TIMEOUT= 8 /* Wait until timeout occurs. Value of timeout can be */ + /* obtained from mysql_get_timeout_value() */ +} MYSQL_ASYNC_STATUS; + Why have enum's instead of defines ? (As these are bits and you are not going to use 'switch' on these, defines makes more sense). #ifdef USE_OLD_FUNCTIONS MYSQL * STDCALL mysql_connect(MYSQL *mysql, const char *host, const char *user, const char *passwd); +int STDCALL mysql_connect_start(MYSQL **ret, MYSQL *mysql, + const char *host, const char *user, + const char *passwd); +int STDCALL mysql_connect_cont(MYSQL **ret, MYSQL *mysql, + int status); int STDCALL mysql_create_db(MYSQL *mysql, const char *DB); +int STDCALL mysql_create_db_start(int *ret, MYSQL *mysql, + const char *DB); +int STDCALL mysql_create_db_cont(int *ret, MYSQL *mysql, + int status); +int STDCALL mysql_drop_db(MYSQL *mysql, const char *DB); +int STDCALL mysql_drop_db_start(int *ret, MYSQL *mysql, + const char *DB); +int STDCALL mysql_drop_db_cont(int *ret, MYSQL *mysql, int status); int STDCALL mysql_drop_db(MYSQL *mysql, const char *DB); #define mysql_reload(mysql) mysql_refresh((mysql),REFRESH_GRANT) #endif If USE_OLD_FUNCTIONS is defined, you don't have to provide the new api. These are deprecated functions that we don't want people to use so there is no reason to provide 'better' versions of them... === modified file 'include/sql_common.h' --- include/sql_common.h 2010-04-01 09:04:26 +0000 +++ include/sql_common.h 2011-09-20 10:49:25 +0000 @@ -26,11 +26,18 @@ extern const char *cant_connect_sqlstate; extern const char *not_error_sqlstate; + +struct mysql_async_context; + struct st_mysql_options_extention { char *plugin_dir; char *default_auth; }; +struct st_mysql_extension { + struct mysql_async_context *async_context; +}; + Why provide 'st_mysql_extension' in a public file? Isn't it enough to have this only in a local file in the client directory? (We don't want anyone to access this structure in any clients) === modified file 'include/violite.h' --- include/violite.h 2010-01-29 10:42:31 +0000 +++ include/violite.h 2011-09-20 10:49:25 +0000 @@ -194,6 +194,8 @@ char *read_pos; /* start of unfetched data in the read buffer */ char *read_end; /* end of unfetched data */ + struct mysql_async_context *async_context; /* For non-blocking API */ + uint read_timeout, write_timeout; /* function pointers. They are similar for socket/SSL/whatever */ void (*viodelete)(Vio*); int (*vioerrno)(Vio*); Hm, I haven't thought about that it's safe to add new things to struct st_vio without breaking the interface. That's good and useful to know! <cut> <cut> === added file 'mysql-test/t/non_blocking_api.test' --- mysql-test/t/non_blocking_api.test 1970-01-01 00:00:00 +0000 +++ mysql-test/t/non_blocking_api.test 2011-09-20 10:49:25 +0000 @@ -0,0 +1,18 @@ +# Test mixing the use of blocking and non-blocking API in a single connection. + +--enable_non_blocking_api +connect (con_nonblock,localhost,root,,test); +--disable_non_blocking_api +connect (con_normal,localhost,root,,test); + Usually all test starts with drop table if exists t1; This is mostly useful when you want to run the tests against an external server. Not critical but good practice to do... <cut> === added file 'mysys/my_context.c' --- mysys/my_context.c 1970-01-01 00:00:00 +0000 +++ mysys/my_context.c 2011-09-20 10:49:25 +0000 @@ -0,0 +1,749 @@ +/* + Copyright 2011 Kristian Nielsen + + Experiments with non-blocking libmysql. + Add the information text after the copyright. (Is the comment still relevant?) + This is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 2 of the License, or + (at your option) any later version. <cut> + Implementation of async context spawning using Posix ucontext and + swapcontext(). +*/ + +#include <stdio.h> +#include <errno.h> + You can remove the above as mysys_priv.h will include these anyway. +#include "mysys_priv.h" +#include "my_context.h" + +#ifdef HAVE_VALGRIND_VALGRIND_H +#include <valgrind/valgrind.h> +#endif Shouldn't this also be depending on the HAVE_valgrind define? <cut> +int +my_context_init(struct my_context *c, size_t stack_size) +{ + if (2*sizeof(int) < sizeof(void *)) + { + fprintf(stderr, + "Error: Unable to store pointer in 2 ints on this architecture\n"); + return -1; + } Please make the above a compile time error instead. #if SIZEOF_CHARP < SIZEOF_INT*2 #error Error: Unable to store pointer in 2 ints on this architecture #endif I think it would be a good idea to do here: bzero(*c, sizeof(*c)); This is better than assuming that 'c' is zeroed on entry. + if (!(c->stack= malloc(stack_size))) + return -1; /* Out of memory */ + c->stack_size= stack_size; +#ifdef HAVE_VALGRIND_VALGRIND_H + c->valgrind_stack_id= + VALGRIND_STACK_REGISTER(c->stack, ((unsigned char *)(c->stack))+stack_size); +#endif +#ifndef DBUG_OFF + c->dbug_state= NULL; +#endif If we do the bzero, you can remove the above + return 0; +} <cut> +int +my_context_init(struct my_context *c, size_t stack_size) +{ + if (!(c->stack_bot= malloc(stack_size))) + return -1; /* Out of memory */ Add bzero of 'c' here. <cut> +int +my_context_init(struct my_context *c, size_t stack_size) +{ Add bzero() here. <cut> +int +my_context_init(struct my_context *c, size_t stack_size) +{ Add bzero here. +#ifndef DBUG_OFF + c->dbug_state= NULL; +#endif + c->lib_fiber= CreateFiber(stack_size, my_context_trampoline, c); + if (c->lib_fiber) + return 0; + else + return -1; Remove else or do: return c->lib_fiber ? 0 : -1; +} <cut> === modified file 'sql-common/client.c' --- sql-common/client.c 2011-03-18 15:03:43 +0000 +++ sql-common/client.c 2011-09-20 10:49:25 +0000 @@ -108,6 +108,7 @@ #include "client_settings.h" #include <sql_common.h> #include <mysql/client_plugin.h> +#include "my_context.h" Should probably be: #include <my_context.h> +#define mysql_extension_get(MYSQL, X) \ + ((MYSQL)->extension ? (MYSQL)->extension->X : NULL) +#define mysql_extension_set(MYSQL, X, VAL) \ + if (!(MYSQL)->extension) \ + (MYSQL)->extension= (struct st_mysql_extension *) \ + my_malloc(sizeof(struct st_mysql_extension), \ + MYF(MY_WME | MY_ZEROFILL)); \ + (MYSQL)->extension->X= VAL; + In 5.3 and 5.5 we have the following macro: #define extension_set(OPTS, X, VAL) \ if (!(OPTS)->extension) \ (OPTS)->extension= (struct st_mysql_options_extention *) \ my_malloc(sizeof(struct st_mysql_options_extention), \ MYF(MY_WME | MY_ZEROFILL)); \ (OPTS)->extension->X= VAL; Why not also store your new extension in mysql->option->extension instead of mysql->extensions ? This would allow you to reuse the existing macros.. +/* + Fetch the context for asynchronous API calls, allocating a new one if + necessary. +*/ +#define STACK_SIZE (4096*15) + +struct mysql_async_context * +mysql_get_async_context(MYSQL *mysql) +{ + struct mysql_async_context *b; + if ((b= mysql_extension_get(mysql, async_context))) + return b; + + if (!(b= (struct mysql_async_context *) + my_malloc(sizeof(*b), MYF(MY_ZEROFILL)))) + { + set_mysql_error(mysql, CR_OUT_OF_MEMORY, unknown_sqlstate); + return NULL; + } Remove the MY_ZEROFILL and let my_context_init() do it. (Makes the function cleaner). + if (my_context_init(&b->async_context, STACK_SIZE)) + { + my_free(b, MYF(0)); + return NULL; + } <cut> /************************************************************************** Get column lengths of the current row @@ -2537,6 +2577,26 @@ } +static int +connect_sync_or_async(MYSQL *mysql, NET *net, my_socket fd, + const struct sockaddr *name, uint namelen) +{ + extern int my_connect_async(struct mysql_async_context *b, my_socket fd, + const struct sockaddr *name, uint namelen, + uint timeout); Please move the above definitioon to header file that is also read by the file the defines the function. (Otherwise someone can change the function without you getting a compiler error). + struct mysql_async_context *actxt= mysql_extension_get(mysql, async_context); + + if (actxt && actxt->active) + { + my_bool old_mode; + vio_blocking(net->vio, FALSE, &old_mode); + return my_connect_async(actxt, fd, name, namelen, + mysql->options.connect_timeout); + } + else Remove else + return my_connect(fd, name, namelen, mysql->options.connect_timeout); +} + <cut> my_bool mysql_reconnect(MYSQL *mysql) { <cut> + if ((ctxt= mysql_extension_get(mysql, async_context)) && ctxt->active) + { + hook_data.orig_mysql= mysql; + hook_data.new_mysql= &tmp_mysql; + hook_data.orig_vio= mysql->net.vio; + my_context_install_suspend_resume_hook(ctxt, my_suspend_hook, &hook_data); + } if (!mysql_real_connect(&tmp_mysql,mysql->host,mysql->user,mysql->passwd, mysql->db, mysql->port, mysql->unix_socket, mysql->client_flag | CLIENT_REMEMBER_OPTIONS)) { + if (ctxt) + my_context_install_suspend_resume_hook(ctxt, NULL, NULL); Change the above to: res= mysql_real_connect(&tmp_mysql,mysql->host,mysql->user,mysql->passwd, mysql->db, mysql->port, mysql->unix_socket, mysql->client_flag | CLIENT_REMEMBER_OPTIONS); if (ctxt) my_context_install_suspend_resume_hook(ctxt, NULL, NULL); if (!res) .... This way you only need to call my_context_install_suspend_resume_hook(ctxt, NULL, NULL) once. mysql->net.last_errno= tmp_mysql.net.last_errno; strmov(mysql->net.last_error, tmp_mysql.net.last_error); strmov(mysql->net.sqlstate, tmp_mysql.net.sqlstate); @@ -3109,13 +3220,18 @@ if (mysql_set_character_set(&tmp_mysql, mysql->charset->csname)) { DBUG_PRINT("error", ("mysql_set_character_set() failed")); + tmp_mysql.extension= NULL; bzero((char*) &tmp_mysql.options,sizeof(tmp_mysql.options)); mysql_close(&tmp_mysql); + if (ctxt) + my_context_install_suspend_resume_hook(ctxt, NULL, NULL); Please remove the above call. mysql->net.last_errno= tmp_mysql.net.last_errno; strmov(mysql->net.last_error, tmp_mysql.net.last_error); strmov(mysql->net.sqlstate, tmp_mysql.net.sqlstate); DBUG_RETURN(1); } + if (ctxt) + my_context_install_suspend_resume_hook(ctxt, NULL, NULL); Please remove the above call. <cut> +static void +mysql_close_free_extension(MYSQL *mysql) +{ + if (mysql->extension) + { + if (mysql->extension->async_context) + { + my_context_destroy(&mysql->extension->async_context->async_context); + my_free(mysql->extension->async_context, MYF(0)); + } + my_free(mysql->extension, MYF(0)); + mysql->extension= NULL; + } +} I still think it would be better to use mysql->option.extension as in this case you can do the above with 2 lines of code and we would not need another function. <discussions over IRC about this....> The reason for not using mysql->options.extension was only becasue mysql_real_connect() freed the options on failure. This code is not needed as they will be freed by mysql_close() anyway. So I suggest you remove the code: if (!(client_flag & CLIENT_REMEMBER_OPTIONS)) mysql_close_free_options(mysql); from mysql_real_connect() and change to use mysql->options.extension instead static void mysql_close_free(MYSQL *mysql) { my_free((uchar*) mysql->host_info,MYF(MY_ALLOW_ZERO_PTR)); @@ -3304,6 +3439,33 @@ (As some clients call this after mysql_real_connect() fails) */ +/* + mysql_close() can actually block, at least in theory, if the socket buffer + is full when sending the COM_QUIT command. + + On the other hand, the latter part of mysql_close() needs to free the stack + used for non-blocking operation of blocking stuff, so that later part can + _not_ be done non-blocking. + + Therefore, mysql_pre_close() is used to run the parts of mysql_close() that + may block. It can be called before mysql_close(), and in that case + mysql_close() is guaranteed not to need to block. +*/ Consider changing the name to 'mysql_slow_part_of_close()' and make it static if possible. +void mysql_pre_close(MYSQL *mysql) +{ + if (!mysql) + return; Remove test; impossible with current code, see belove in mysql_close. + /* If connection is still up, send a QUIT message */ + if (mysql->net.vio != 0) + { + free_old_query(mysql); + mysql->status=MYSQL_STATUS_READY; /* Force command */ + mysql->reconnect=0; + simple_command(mysql,COM_QUIT,(uchar*) 0,0,1); + end_server(mysql); /* Sets mysql->net.vio= 0 */ + } +} + void STDCALL mysql_close(MYSQL *mysql) { DBUG_ENTER("mysql_close"); @@ -3311,16 +3473,9 @@ + +my_socket STDCALL +mysql_get_socket(const MYSQL *mysql) +{ + if (mysql->net.vio) + return mysql->net.vio->sd; + else remove else + return INVALID_SOCKET; +} +++ sql-common/mysql_async.c 2011-09-20 10:49:25 +0000 +#ifdef __WIN__ +/* + Windows does not support MSG_DONTWAIT for send()/recv(). So we need to ensure + that the socket is non-blocking at the start of every operation. +*/ +#define WIN_SET_NONBLOCKING(mysql) { \ + my_bool old_mode__; \ + if ((mysql)->net.vio) vio_blocking((mysql)->net.vio, FALSE, &old_mode__); \ + } old_mode__ -> old_mode +#else +#define WIN_SET_NONBLOCKING(mysql) +#endif + +extern struct mysql_async_context *mysql_get_async_context(MYSQL *mysql); Move to header file. +/* Asynchronous connect(); socket must already be set non-blocking. */ +int +my_connect_async(struct mysql_async_context *b, my_socket fd, + const struct sockaddr *name, uint namelen, uint timeout) +{ + int res; +#ifdef __WIN__ + int s_err_size; +#else + socklen_t s_err_size; +#endif It would be nice to just define socklen_t for windows, but this isn't critical. You could have a DBUG_ASSERT here to check if socket is really in non blocking. + + /* + Start to connect asynchronously. + If this will block, we suspend the call and return control to the + application context. The application will then resume us when the socket + polls ready for write, indicating that the connection attempt completed. + */ + res= connect(fd, name, namelen); +#ifdef __WIN__ + if (res != 0) + { + int wsa_err= WSAGetLastError(); + if (wsa_err != WSAEWOULDBLOCK) + return res; +#else + if (res < 0) To not have only one { inside the #ifdef / #else I suggest that you move if (res != 0) { before the +#ifdef __WIN__. This works on both windows and Unix This will make the number of { balance + { + if (errno != EINPROGRESS && errno != EALREADY && errno != EAGAIN) + return res; +#endif + b->timeout_value= timeout; + b->ret_status= MYSQL_WAIT_WRITE | + (timeout ? MYSQL_WAIT_TIMEOUT : 0); +#ifdef __WIN__ + b->ret_status|= MYSQL_WAIT_EXCEPT; +#endif How about setting b->ret_status to 0 before res= connect() ? Then you can move the above code to the previous #ifdef __WIN__ and change the setting of b->ret_status to use |= This removes the extra ifdef __WIN__ and also ensures that ret_status is 0 if evertything went well. (not required but still better than a random value when trying to understand what's going in). + if (b->suspend_resume_hook) + (*b->suspend_resume_hook)(TRUE, b->suspend_resume_hook_user_data); + my_context_yield(&b->async_context); + if (b->suspend_resume_hook) + (*b->suspend_resume_hook)(FALSE, b->suspend_resume_hook_user_data); + if (b->ret_status & MYSQL_WAIT_TIMEOUT) + return -1; + + s_err_size= sizeof(int); Better to use (safer and more descriptive): s_err_size= sizeof(res); + if (getsockopt(fd, SOL_SOCKET, SO_ERROR, (char*) &res, &s_err_size) != 0) + return -1; <cut> + +ssize_t +my_recv_async(struct mysql_async_context *b, int fd, + unsigned char *buf, size_t size, uint timeout) +{ + ssize_t res; + + for (;;) + { + res= recv(fd, buf, size, +#ifdef __WIN__ + 0 +#else + MSG_DONTWAIT +#endif + ); Better to do before function start: #ifdef __WIN__ #define MSG_DONTWAIT 0 #endif and remove the above #ifdef in middle of call. Another option is to use the IF_WIN() macro: IF_WIN(0, MSG_DONTWAIT) + if (res >= 0 || +#ifdef __WIN__ + WSAGetLastError() != WSAEWOULDBLOCK +#else + (errno != EAGAIN && errno != EINTR) +#endif Please define macro: #define IS_BLOCKING_ERROR IF_WIN(WSAGetLastError() != WSAEWOULDBLOCK, (errno != EAGAIN && errno != EINTR)) and use that instead of having an ifdef in the middle of the code. + ) + return res; + b->ret_status= MYSQL_WAIT_READ; + if (timeout) + { + b->ret_status|= MYSQL_WAIT_TIMEOUT; + b->timeout_value= timeout; + } + if (b->suspend_resume_hook) + (*b->suspend_resume_hook)(TRUE, b->suspend_resume_hook_user_data); + my_context_yield(&b->async_context); + if (b->suspend_resume_hook) + (*b->suspend_resume_hook)(FALSE, b->suspend_resume_hook_user_data); The following call was quite confusion as we are checking a variable that we where just setting. <phone call> The reasons is that b->ret_status is both an in an out parameter. We agree to do one of the following to make the code easier to understand: - Change to use different bits for in and out status. - Change to use different variables for in and out status. + if (b->ret_status & MYSQL_WAIT_TIMEOUT) + return -1; + } +} + Add empty row here +ssize_t +my_send_async(struct mysql_async_context *b, int fd, + const unsigned char *buf, size_t size, uint timeout) +{ + ssize_t res; + + for (;;) + { + res= send(fd, buf, size, +#ifdef __WIN__ + 0 +#else + MSG_DONTWAIT +#endif + ); + if (res >= 0 || +#ifdef __WIN__ + WSAGetLastError() != WSAEWOULDBLOCK +#else + (errno != EAGAIN && errno != EINTR) +#endif + ) See comments for previous function how to get rid of the ifdefs' <cut> +#ifdef HAVE_OPENSSL +int +my_ssl_read_async(struct mysql_async_context *b, SSL *ssl, + void *buf, int size) +{ + int res, ssl_err; + + for (;;) + { + res= SSL_read(ssl, buf, size); + if (res >= 0) + return res; + ssl_err= SSL_get_error(ssl, res); + if (ssl_err == SSL_ERROR_WANT_READ) + b->ret_status= MYSQL_WAIT_READ; + else if (ssl_err == SSL_ERROR_WANT_WRITE) + b->ret_status= MYSQL_WAIT_WRITE; + else + return res; + if (b->suspend_resume_hook) + (*b->suspend_resume_hook)(TRUE, b->suspend_resume_hook_user_data); + my_context_yield(&b->async_context); + if (b->suspend_resume_hook) + (*b->suspend_resume_hook)(FALSE, b->suspend_resume_hook_user_data); + } +} + +int +my_ssl_write_async(struct mysql_async_context *b, SSL *ssl, + const void *buf, int size) +{ + int res, ssl_err; + + for (;;) + { + res= SSL_write(ssl, buf, size); + if (res >= 0) + return res; + ssl_err= SSL_get_error(ssl, res); + if (ssl_err == SSL_ERROR_WANT_READ) + b->ret_status= MYSQL_WAIT_READ; + else if (ssl_err == SSL_ERROR_WANT_WRITE) + b->ret_status= MYSQL_WAIT_WRITE; + else + return res; + if (b->suspend_resume_hook) + (*b->suspend_resume_hook)(TRUE, b->suspend_resume_hook_user_data); + my_context_yield(&b->async_context); + if (b->suspend_resume_hook) + (*b->suspend_resume_hook)(FALSE, b->suspend_resume_hook_user_data); + } +} 90 % of the above functions are the same: I would do it this way: static my_bool my_ssl_async_check_result(int *res, struct mysql_async_context *b) { int ssl_err; if (*res >= 0) return 1; ssl_err= SSL_get_error(ssl, *res); if (ssl_err == SSL_ERROR_WANT_READ) b->ret_status= MYSQL_WAIT_READ; else if (ssl_err == SSL_ERROR_WANT_WRITE) b->ret_status= MYSQL_WAIT_WRITE; else return 1; if (b->suspend_resume_hook) (*b->suspend_resume_hook)(TRUE, b->suspend_resume_hook_user_data); my_context_yield(&b->async_context); if (b->suspend_resume_hook) (*b->suspend_resume_hook)(FALSE, b->suspend_resume_hook_user_data); return 0; } And then: my_ssl_read_async(struct mysql_async_context *b, SSL *ssl, void *buf, int size) { int res; for (;;) { res= SSL_read(ssl, buf, size); if (my_ssl_async_check_result(res, b)) return res; } } <cut> +#endif /* HAVE_OPENSSL */ + +unsigned int STDCALL +mysql_get_timeout_value(const MYSQL *mysql) +{ + if (mysql->extension && mysql->extension->async_context) + return mysql->extension->async_context->timeout_value; + else Remove else + return 0; +} + Add empty line +/* + Now create non-blocking definitions for all the calls that may block. + + Each call FOO gives rise to FOO_start() that prepares the MYSQL object for + doing non-blocking calls that can suspend operation mid-way, and then starts + the call itself. And a FOO_start_internal trampoline to assist with running + the real call in a co-routine that can be suspended. And a FOO_cont() that + can continue a suspended operation. +*/ + +#define MK_ASYNC_CALLS(call__, decl_args__, invoke_args__, cont_arg__, mysql_val__, parms_mysql_val__, parms_assign__, ret_type__, err_val__, ok_val__, extra1__) \ +static void \ +call__ ## _start_internal(void *d) \ +{ \ As discussed on IRC, it would be good to have the functions declarations outside of the macro; This makes it easier to make breakpoints in the code and help debugging. + struct call__ ## _params *parms; \ + ret_type__ ret; \ + struct mysql_async_context *b; \ + \ + parms= (struct call__ ## _params *)d; \ + b= (parms_mysql_val__)->extension->async_context; \ + \ + ret= call__ invoke_args__; \ + b->ret_result. ok_val__ = ret; \ + b->ret_status= 0; \ +} \ +int STDCALL \ +call__ ## _start decl_args__ \ +{ \ + int res; \ + struct mysql_async_context *b; \ + struct call__ ## _params parms; \ + \ + extra1__ \ + if (!(b= mysql_get_async_context((mysql_val__)))) \ + { \ + *ret= err_val__; \ + return 0; \ + } \ Instead of calling mysql_get_async_context, another option would be to do: mysql_options(mysql, MYSQL_OPT_ASYNC_IO, (char*) &STACK_SIZE) And if STACK_SIZE is 0, then we allocate our default stack size then you can change the above code to: b= mysql_val__->options.extension->async_context; If it doesn't exist, it's ok to crash on access. (Same as if mysql is 0) + parms_assign__ \ + \ + b->active= 1; \ + res= my_context_spawn(&b->async_context, call__ ## _start_internal, &parms);\ + b->active= 0; \ Set also b->suspended= 0 here. (Makes the rest of the code shorter) + if (res < 0) \ + { \ + set_mysql_error((mysql_val__), CR_OUT_OF_MEMORY, unknown_sqlstate); \ How do we know the reason is out of memory? (Just curious) + b->suspended= 0; \ + *ret= err_val__; \ + return 0; \ + } \ + else if (res > 0) \ Remove else + { \ + /* Suspended. */ \ + b->suspended= 1; \ + return b->ret_status; \ + } \ + else \ Remove else and {} + { \ + /* Finished. */ \ + b->suspended= 0; \ + *ret= b->ret_result. ok_val__; \ + return 0; \ + } \ +} \ If you change to test for (res > 0) first, you can combine the two return 0 to one row and thus remove some of the { }. Makes the code shorter (at least to read): b->active= b->suspended= 0; if (res > 0) { b->suspended= 1; return b->ret_status; } if (res < 0) { set_mysql_error((mysql_val__), CR_OUT_OF_MEMORY, unknown_sqlstate); *ret= err_val__; } else *ret= b->ret_result. ok_val__; /* Finished. */ return 0; +int STDCALL \ +call__ ## _cont(ret_type__ *ret, cont_arg__, int ready_status) \ +{ \ + int res; \ + struct mysql_async_context *b; \ + \ + b= (mysql_val__)->extension->async_context; \ + if (!b || !b->suspended) \ Remove test for !b; You can assume it's exists. (We can never protect aginst wrong usage) + { \ + set_mysql_error((mysql_val__), CR_COMMANDS_OUT_OF_SYNC, unknown_sqlstate);\ + *ret= err_val__; \ + return 0; \ + } \ + \ + b->active= 1; \ + b->ret_status= ready_status; \ + res= my_context_continue(&b->async_context); \ + b->active= 0; \ + if (res < 0) \ + { \ + set_mysql_error((mysql_val__), CR_OUT_OF_MEMORY, unknown_sqlstate); \ + b->suspended= 0; \ + *ret= err_val__; \ + return 0; \ + } \ + else if (res > 0) \ + { \ + /* Suspended. */ \ + return b->ret_status; \ + } \ + else \ + { \ + /* Finished. */ \ + b->suspended= 0; \ + *ret= b->ret_result. ok_val__; \ + return 0; \ + } \ You can do the same 'suspend and check for error' optimization I did in previous function to reduce the above code with 4 lines. +} + +#define MK_ASYNC_CALLS_VOID_RETURN(call__, decl_args__, invoke_args__, cont_arg__, mysql_val__, parms_mysql_val__, parms_assign__, extra1__) \ +static void \ +call__ ## _start_internal(void *d) \ +{ \ + struct call__ ## _params *parms; \ + struct mysql_async_context *b; \ + \ + parms= (struct call__ ## _params *)d; \ + b= (parms_mysql_val__)->extension->async_context; \ + \ + call__ invoke_args__; \ + b->ret_status= 0; \ +} \ +int STDCALL \ +call__ ## _start decl_args__ \ +{ \ + int res; \ + struct mysql_async_context *b; \ + struct call__ ## _params parms; \ + \ + extra1__ \ + if (!(b= mysql_get_async_context((mysql_val__)))) \ + { \ + return 0; \ + } \ You can remove test if we know that b always exists. At least you can remove '{' + parms_assign__ \ + \ + b->active= 1; \ + res= my_context_spawn(&b->async_context, call__ ## _start_internal, &parms);\ + b->active= 0; \ + if (res < 0) \ + { \ + set_mysql_error((mysql_val__), CR_OUT_OF_MEMORY, unknown_sqlstate); \ + b->suspended= 0; \ + return 0; \ + } \ + else if (res > 0) \ + { \ + /* Suspended. */ \ + b->suspended= 1; \ + return b->ret_status; \ + } \ + else \ + { \ + /* Finished. */ \ + b->suspended= 0; \ + return 0; \ + } \ +} \ See previous comment of how to remove 4 code lines. +int STDCALL \ +call__ ## _cont(cont_arg__, int ready_status) \ +{ \ + int res; \ + struct mysql_async_context *b; \ + \ + b= (mysql_val__)->extension->async_context; \ + if (!b || !b->suspended) \ Assume !b exists + { \ + set_mysql_error((mysql_val__), CR_COMMANDS_OUT_OF_SYNC, unknown_sqlstate);\ + return 0; \ + } \ + \ + b->active= 1; \ + b->ret_status= ready_status; \ + res= my_context_continue(&b->async_context); \ + b->active= 0; \ + if (res < 0) \ + { \ + set_mysql_error((mysql_val__), CR_OUT_OF_MEMORY, unknown_sqlstate); \ + b->suspended= 0; \ + return 0; \ + } \ + else if (res > 0) \ + { \ + /* Suspended. */ \ + return b->ret_status; \ + } \ + else \ + { \ + /* Finished. */ \ + b->suspended= 0; \ + return 0; \ + } \ +} Remove extra lines above. Add comment here (and for other similar structs): /* This struct is used by the next MK_ASYNC_CALL */ + +struct mysql_real_connect_params { + MYSQL *mysql; + const char *host; + const char *user; + const char *passwd; + const char *db; + unsigned int port; + const char *unix_socket; + unsigned long client_flags; +}; +MK_ASYNC_CALLS( + mysql_real_connect, + (MYSQL **ret, MYSQL *mysql, const char *host, const char *user, + const char *passwd, const char *db, unsigned int port, + const char *unix_socket, unsigned long client_flags), + (parms->mysql, parms->host, parms->user, parms->passwd, parms->db, + parms->port, parms->unix_socket, parms->client_flags), <cut> +mysql_close_start(MYSQL *sock) +{ + int res; + + /* It is legitimate to have NULL sock argument, which will do nothing. */ + if (sock) + { + res= mysql_pre_close_start(sock); + /* If we need to block, return now and do the rest in mysql_close_cont(). */ + if (res) + return res; + } + mysql_close(sock); + return 0; +} +int STDCALL +mysql_close_cont(MYSQL *sock, int ready_status) +{ + int res; + + res= mysql_pre_close_cont(sock, ready_status); + if (res) + return res; + mysql_close(sock); + return 0; +} You should use the above mysql_close functions also in client/async_example.cc +#ifdef USE_OLD_FUNCTIONS Forget these! We should remove them soon anyway... <cut> === added file 'tests/async_queries.c' +void +add_query(const char *q) +{ + struct query_entry *e; + char *q2; + size_t len; + + e= malloc(sizeof(*e)); + q2= strdup(q); Would be better if you used my_malloc / my_strdup here. (To get things safemalloc checked). + if (!e || !q2) + fatal(NULL, "Out of memory"); + + /* Remove any trailing newline. */ + len= strlen(q2); + if (q2[len] == '\n') + q2[len--]= '\0'; + if (q2[len] == '\r') + q2[len--]= '\0'; + + e->next= NULL; + e->query= q2; + e->index= query_counter++; + *tail_ptr= e; + tail_ptr= &e->next; +} <cut> === modified file 'vio/viosocket.c' --- vio/viosocket.c 2011-05-14 16:42:07 +0000 +++ vio/viosocket.c 2011-09-20 10:49:25 +0000 @@ -21,6 +21,7 @@ */ #include "vio_priv.h" +#include "my_context.h" int vio_errno(Vio *vio __attribute__((unused))) { @@ -31,18 +32,34 @@ size_t vio_read(Vio * vio, uchar* buf, size_t size) { size_t r; + extern ssize_t my_recv_async(struct mysql_async_context *b, int fd, + unsigned char *buf, size_t size, uint timeout); Move to include file. DBUG_ENTER("vio_read"); DBUG_PRINT("enter", ("sd: %d buf: 0x%lx size: %u", vio->sd, (long) buf, (uint) size)); /* Ensure nobody uses vio_read_buff and vio_read simultaneously */ DBUG_ASSERT(vio->read_end == vio->read_pos); + if (vio->async_context && vio->async_context->active) + r= my_recv_async(vio->async_context, vio->sd, buf, size, vio->read_timeout); + else + { <cut> === modified file 'vio/viossl.c' @@ -21,6 +21,7 @@ */ #include "vio_priv.h" +#include "my_context.h" #ifdef HAVE_OPENSSL @@ -90,11 +91,16 @@ size_t vio_ssl_read(Vio *vio, uchar* buf, size_t size) { size_t r; + extern int my_ssl_read_async(struct mysql_async_context *b, SSL *ssl, + void *buf, int size); Move to extern file (Same for other extern declarations in this file). <cut> Good and intresting work! Regards, Monty

4 9

[Maria-developers] MYSQL_THDVAR and SET GLOBAL
by MARK CALLAGHAN 30 Nov '11

30 Nov '11

Is there an easy way to prevent SET GLOBAL ... from being done for plugin variables declared via MYSQL_THDVAR_...? For the following variable I can do SET GLOBAL innodb_fake_changes=1 and that sets a global variable. static MYSQL_THDVAR_BOOL(fake_changes, PLUGIN_VAR_OPCMDARG, "In the transaction after enabled, UPDATE, INSERT and DELETE only move the cursor to the records " "and do nothing other operations (no changes, no ibuf, no undo, no transaction log) in the transaction. " "This is to cause replication prefetch IO. ATTENTION: the transaction started after enabled is affected.", check_fake_changes, NULL, FALSE); -- Mark Callaghan mdcallag(a)gmail.com

3 2

[Maria-developers] MariaDB 5.5 binary tarball packages and libaio
by Kristian Nielsen 30 Nov '11

30 Nov '11

Hi Wlad, I got some progress with Buildbot and producing binary tarball packages for MariaDB 5.5, using your suggestions. But I encountered a problem. I used -DBUILD_CONFIG=mysql-release and installed libaio-dev. But then the server fails like this: ./bin/mysqld: error while loading shared libraries: libaio.so.1: cannot open shared object file: No such file or directory Of course - as libaio.so must be installed by users. I do not think it is appropriate that users need to install libaio.so to use the binary tarballs (as those are supposed to work anywhere - on the other hand, for .deb for example we _should_ use libaio of course, and dependencies will take care that it is installed). So what to do? For now I've tried to build without libaio for the generic tarballs (-DIGNORE_AIO_CHECK=1). So binaries will work; to get aio, users can build themselves or install real packages. Or do you have another idea? Do you know what MySQL does? One option could be statically linking libaio. Now in general, static linking is a can of worms with many pitfalls, however for just libaio it might work (libaio.so is statically linked on my Debian Wheezy). However, I am not sure if this is safe in general, nor do I know how to make cmake link just libaio statically? Any ideas? - Kristian.

4 4

Re: [Maria-developers] [Commits] Rev 3322: Fixed bug lp:825051 in file:///home/tsk/mprog/src/5.3/
by Oleksandr Byelkin 30 Nov '11

30 Nov '11

29.11.2011 23:06, timour(a)askmonty.org пишет: > At file:///home/tsk/mprog/src/5.3/ > > ------------------------------------------------------------ > revno: 3322 > revision-id: timour(a)askmonty.org-20111129210639-bjcn2pfocub6uwba > parent: holyfoot(a)askmonty.org-20111129112752-6cy8wy7h5q58l6je > fixes bug(s): https://launchpad.net/bugs/825051 > committer: timour(a)askmonty.org > branch nick: 5.3 > timestamp: Tue 2011-11-29 23:06:39 +0200 > message: > Fixed bug lp:825051 > > The cause of the wrong result was that Item_ref_null_helper::get_date() > didn't use a method of the *_result() family, and fetched the data > for the field from the current row instead of result_field. Changed to > use the correct *_result() method, like to all other similar methods > of Item_ref_null_helper. OK to push. Do not forget that we agreed to fix this typo in 5.2 at least.

1 0

[Maria-developers] Alternate fix for BUG#833777 and BUG#894397
by Sergei Petrunia 28 Nov '11

28 Nov '11

Hi Timour, Please find it below. Any objections? === modified file 'sql/sql_select.cc' --- sql/sql_select.cc 2011-11-25 19:54:36 +0000 +++ sql/sql_select.cc 2011-11-28 11:40:42 +0000 @@ -14776,14 +14776,10 @@ do_select(JOIN *join,List<Item> *fields, /* HAVING will be checked after processing aggregate functions, But WHERE should checkd here (we alredy have read tables). - If there is join->exec_const_cond, and all tables are constant, then it - is equivalent to join->conds. exec_const_cond is already checked in the - beginning of JOIN::exec. If it is false, JOIN::exec returns zero - result already there, therefore execution reaches this point only if - exec_const_cond is TRUE. Since it is equvalent to join->conds, then - join->conds is also TRUE. + + (TODO new comment here) */ - if (!join->conds || join->exec_const_cond || join->conds->val_int()) + if (!join->outer_ref_cond || join->outer_ref_cond->val_int()) { error= (*end_select)(join, 0, 0); if (error == NESTED_LOOP_OK || error == NESTED_LOOP_QUERY_LIMIT) BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] [Commits] Rev 3308: Fixed LP BUG#747278 in file:///home/bell/maria/bzr/work-maria-5.3-lpb747278/
by Sergei Petrunia 28 Nov '11

28 Nov '11

Hi Sanja, Ok to push. On Wed, Nov 23, 2011 at 01:46:22AM +0200, sanja(a)askmonty.org wrote: > At file:///home/bell/maria/bzr/work-maria-5.3-lpb747278/ > > ------------------------------------------------------------ > revno: 3308 > revision-id: sanja(a)askmonty.org-20111122234621-b1t47i7xgzsg1hwo > parent: holyfoot(a)askmonty.org-20111122135733-vgsg077x9rbn08r6 > committer: sanja(a)askmonty.org > branch nick: work-maria-5.3-lpb747278 > timestamp: Wed 2011-11-23 01:46:21 +0200 > message: > Fixed LP BUG#747278 > > The problem was that when we have single row subquery with no rows > Item_cache(es) which represent result row was not null and being > requested via element_index() returned random value. > > The fix is setting all Item_cache(es) in NULL before executing the > query (reset() method) which guaranty NULL value of whole query > or its elements requested in any way if no rows was found. > > set_null() method was added to Item_cache to guaranty correct NULL > value in case of reseting the cache. > === modified file 'mysql-test/r/subselect.result' > --- a/mysql-test/r/subselect.result 2011-11-22 10:06:46 +0000 > +++ b/mysql-test/r/subselect.result 2011-11-22 23:46:21 +0000 > @@ -5681,4 +5681,85 @@ select a from t1 where a in (select a fr > a > 1 > drop table t1; > +# > +# LP BUG#747278 incorrect values of the NULL (no rows) single > +# row subquery requested via element_index() interface > +# > +CREATE TABLE t1 (f1a int, f1b int) ; > +INSERT IGNORE INTO t1 VALUES (1,1),(2,2); > +CREATE TABLE t2 ( f2 int); > +INSERT IGNORE INTO t2 VALUES (3),(4); > +CREATE TABLE t3 (f3a int default 1, f3b int default 2); > +INSERT INTO t3 VALUES (1,1),(2,2); > +set @old_optimizer_switch = @@session.optimizer_switch; > +set @@optimizer_switch='materialization=on,partial_match_rowid_merge=on,partial_match_table_scan=off,subquery_cache=off,semijoin=off'; > +SELECT (SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) FROM t2; > +(SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) FROM t2; > +(SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) FROM t2; > +(SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1); > +(SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) > +NULL > +SELECT (SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) FROM t2; > +(SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) FROM t2; > +(SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) FROM t2; > +(SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1); > +(SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) > +NULL > +set @@session.optimizer_switch=@old_optimizer_switch; > +SELECT (SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) FROM t2; > +(SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) FROM t2; > +(SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) FROM t2; > +(SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1); > +(SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) > +NULL > +SELECT (SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) FROM t2; > +(SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) FROM t2; > +(SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) FROM t2; > +(SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1); > +(SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) > +NULL > +select (null, null) = (null, null); > +(null, null) = (null, null) > +NULL > +SELECT (SELECT f3a, f3a FROM t3 where f3a > 3) = (0, 0); > +(SELECT f3a, f3a FROM t3 where f3a > 3) = (0, 0) > +NULL > +drop tables t1,t2,t3; > +# return optimizer switch changed in the beginning of this test > set optimizer_switch=@subselect_tmp; > > === modified file 'mysql-test/r/subselect_no_mat.result' > --- a/mysql-test/r/subselect_no_mat.result 2011-11-22 10:06:46 +0000 > +++ b/mysql-test/r/subselect_no_mat.result 2011-11-22 23:46:21 +0000 > @@ -5680,6 +5680,87 @@ select a from t1 where a in (select a fr > a > 1 > drop table t1; > +# > +# LP BUG#747278 incorrect values of the NULL (no rows) single > +# row subquery requested via element_index() interface > +# > +CREATE TABLE t1 (f1a int, f1b int) ; > +INSERT IGNORE INTO t1 VALUES (1,1),(2,2); > +CREATE TABLE t2 ( f2 int); > +INSERT IGNORE INTO t2 VALUES (3),(4); > +CREATE TABLE t3 (f3a int default 1, f3b int default 2); > +INSERT INTO t3 VALUES (1,1),(2,2); > +set @old_optimizer_switch = @@session.optimizer_switch; > +set @@optimizer_switch='materialization=on,partial_match_rowid_merge=on,partial_match_table_scan=off,subquery_cache=off,semijoin=off'; > +SELECT (SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) FROM t2; > +(SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) FROM t2; > +(SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) FROM t2; > +(SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1); > +(SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) > +NULL > +SELECT (SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) FROM t2; > +(SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) FROM t2; > +(SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) FROM t2; > +(SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1); > +(SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) > +NULL > +set @@session.optimizer_switch=@old_optimizer_switch; > +SELECT (SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) FROM t2; > +(SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) FROM t2; > +(SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) FROM t2; > +(SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1); > +(SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) > +NULL > +SELECT (SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) FROM t2; > +(SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) FROM t2; > +(SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) FROM t2; > +(SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1); > +(SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) > +NULL > +select (null, null) = (null, null); > +(null, null) = (null, null) > +NULL > +SELECT (SELECT f3a, f3a FROM t3 where f3a > 3) = (0, 0); > +(SELECT f3a, f3a FROM t3 where f3a > 3) = (0, 0) > +NULL > +drop tables t1,t2,t3; > +# return optimizer switch changed in the beginning of this test > set optimizer_switch=@subselect_tmp; > set optimizer_switch=default; > select @@optimizer_switch like '%materialization=on%'; > > === modified file 'mysql-test/r/subselect_no_opts.result' > --- a/mysql-test/r/subselect_no_opts.result 2011-11-21 20:01:47 +0000 > +++ b/mysql-test/r/subselect_no_opts.result 2011-11-22 23:46:21 +0000 > @@ -5676,5 +5676,86 @@ select a from t1 where a in (select a fr > a > 1 > drop table t1; > +# > +# LP BUG#747278 incorrect values of the NULL (no rows) single > +# row subquery requested via element_index() interface > +# > +CREATE TABLE t1 (f1a int, f1b int) ; > +INSERT IGNORE INTO t1 VALUES (1,1),(2,2); > +CREATE TABLE t2 ( f2 int); > +INSERT IGNORE INTO t2 VALUES (3),(4); > +CREATE TABLE t3 (f3a int default 1, f3b int default 2); > +INSERT INTO t3 VALUES (1,1),(2,2); > +set @old_optimizer_switch = @@session.optimizer_switch; > +set @@optimizer_switch='materialization=on,partial_match_rowid_merge=on,partial_match_table_scan=off,subquery_cache=off,semijoin=off'; > +SELECT (SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) FROM t2; > +(SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) FROM t2; > +(SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) FROM t2; > +(SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1); > +(SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) > +NULL > +SELECT (SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) FROM t2; > +(SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) FROM t2; > +(SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) FROM t2; > +(SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1); > +(SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) > +NULL > +set @@session.optimizer_switch=@old_optimizer_switch; > +SELECT (SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) FROM t2; > +(SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) FROM t2; > +(SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) FROM t2; > +(SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1); > +(SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) > +NULL > +SELECT (SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) FROM t2; > +(SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) FROM t2; > +(SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) FROM t2; > +(SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1); > +(SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) > +NULL > +select (null, null) = (null, null); > +(null, null) = (null, null) > +NULL > +SELECT (SELECT f3a, f3a FROM t3 where f3a > 3) = (0, 0); > +(SELECT f3a, f3a FROM t3 where f3a > 3) = (0, 0) > +NULL > +drop tables t1,t2,t3; > +# return optimizer switch changed in the beginning of this test > set optimizer_switch=@subselect_tmp; > set @optimizer_switch_for_subselect_test=null; > > === modified file 'mysql-test/r/subselect_no_scache.result' > --- a/mysql-test/r/subselect_no_scache.result 2011-11-22 10:06:46 +0000 > +++ b/mysql-test/r/subselect_no_scache.result 2011-11-22 23:46:21 +0000 > @@ -5685,6 +5685,87 @@ select a from t1 where a in (select a fr > a > 1 > drop table t1; > +# > +# LP BUG#747278 incorrect values of the NULL (no rows) single > +# row subquery requested via element_index() interface > +# > +CREATE TABLE t1 (f1a int, f1b int) ; > +INSERT IGNORE INTO t1 VALUES (1,1),(2,2); > +CREATE TABLE t2 ( f2 int); > +INSERT IGNORE INTO t2 VALUES (3),(4); > +CREATE TABLE t3 (f3a int default 1, f3b int default 2); > +INSERT INTO t3 VALUES (1,1),(2,2); > +set @old_optimizer_switch = @@session.optimizer_switch; > +set @@optimizer_switch='materialization=on,partial_match_rowid_merge=on,partial_match_table_scan=off,subquery_cache=off,semijoin=off'; > +SELECT (SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) FROM t2; > +(SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) FROM t2; > +(SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) FROM t2; > +(SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1); > +(SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) > +NULL > +SELECT (SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) FROM t2; > +(SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) FROM t2; > +(SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) FROM t2; > +(SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1); > +(SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) > +NULL > +set @@session.optimizer_switch=@old_optimizer_switch; > +SELECT (SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) FROM t2; > +(SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) FROM t2; > +(SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) FROM t2; > +(SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1); > +(SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) > +NULL > +SELECT (SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) FROM t2; > +(SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) FROM t2; > +(SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) FROM t2; > +(SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1); > +(SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) > +NULL > +select (null, null) = (null, null); > +(null, null) = (null, null) > +NULL > +SELECT (SELECT f3a, f3a FROM t3 where f3a > 3) = (0, 0); > +(SELECT f3a, f3a FROM t3 where f3a > 3) = (0, 0) > +NULL > +drop tables t1,t2,t3; > +# return optimizer switch changed in the beginning of this test > set optimizer_switch=@subselect_tmp; > set optimizer_switch=default; > select @@optimizer_switch like '%subquery_cache=on%'; > > === modified file 'mysql-test/r/subselect_no_semijoin.result' > --- a/mysql-test/r/subselect_no_semijoin.result 2011-11-22 10:06:46 +0000 > +++ b/mysql-test/r/subselect_no_semijoin.result 2011-11-22 23:46:21 +0000 > @@ -5676,5 +5676,86 @@ select a from t1 where a in (select a fr > a > 1 > drop table t1; > +# > +# LP BUG#747278 incorrect values of the NULL (no rows) single > +# row subquery requested via element_index() interface > +# > +CREATE TABLE t1 (f1a int, f1b int) ; > +INSERT IGNORE INTO t1 VALUES (1,1),(2,2); > +CREATE TABLE t2 ( f2 int); > +INSERT IGNORE INTO t2 VALUES (3),(4); > +CREATE TABLE t3 (f3a int default 1, f3b int default 2); > +INSERT INTO t3 VALUES (1,1),(2,2); > +set @old_optimizer_switch = @@session.optimizer_switch; > +set @@optimizer_switch='materialization=on,partial_match_rowid_merge=on,partial_match_table_scan=off,subquery_cache=off,semijoin=off'; > +SELECT (SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) FROM t2; > +(SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) FROM t2; > +(SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) FROM t2; > +(SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1); > +(SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) > +NULL > +SELECT (SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) FROM t2; > +(SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) FROM t2; > +(SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) FROM t2; > +(SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1); > +(SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) > +NULL > +set @@session.optimizer_switch=@old_optimizer_switch; > +SELECT (SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) FROM t2; > +(SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) FROM t2; > +(SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) FROM t2; > +(SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1); > +(SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) > +NULL > +SELECT (SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) FROM t2; > +(SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) FROM t2; > +(SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) FROM t2; > +(SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) > +NULL > +NULL > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1); > +(SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) > +NULL > +select (null, null) = (null, null); > +(null, null) = (null, null) > +NULL > +SELECT (SELECT f3a, f3a FROM t3 where f3a > 3) = (0, 0); > +(SELECT f3a, f3a FROM t3 where f3a > 3) = (0, 0) > +NULL > +drop tables t1,t2,t3; > +# return optimizer switch changed in the beginning of this test > set optimizer_switch=@subselect_tmp; > set @optimizer_switch_for_subselect_test=null; > > === modified file 'mysql-test/t/subselect.test' > --- a/mysql-test/t/subselect.test 2011-11-21 20:16:01 +0000 > +++ b/mysql-test/t/subselect.test 2011-11-22 23:46:21 +0000 > @@ -4778,4 +4778,47 @@ set @@optimizer_switch='in_to_exists=on, > select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1))))))))))))))))))))))))))))); > drop table t1; > > +--echo # > +--echo # LP BUG#747278 incorrect values of the NULL (no rows) single > +--echo # row subquery requested via element_index() interface > +--echo # > + > +CREATE TABLE t1 (f1a int, f1b int) ; > +INSERT IGNORE INTO t1 VALUES (1,1),(2,2); > +CREATE TABLE t2 ( f2 int); > +INSERT IGNORE INTO t2 VALUES (3),(4); > +CREATE TABLE t3 (f3a int default 1, f3b int default 2); > +INSERT INTO t3 VALUES (1,1),(2,2); > + > +# check different IN with switches where the bug was found > +set @old_optimizer_switch = @@session.optimizer_switch; > +set @@optimizer_switch='materialization=on,partial_match_rowid_merge=on,partial_match_table_scan=off,subquery_cache=off,semijoin=off'; > + > +SELECT (SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) FROM t2; > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) FROM t2; > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) FROM t2; > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1); > +SELECT (SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) FROM t2; > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) FROM t2; > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) FROM t2; > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1); > + > +set @@session.optimizer_switch=@old_optimizer_switch; > + > +# check different IN with default switches > +SELECT (SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a FROM t1) FROM t2; > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a,f1a FROM t1) FROM t2; > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1) FROM t2; > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) NOT IN (SELECT f1a, f1b FROM t1); > +SELECT (SELECT f3a FROM t3 where f3a > 3) IN (SELECT f1a FROM t1) FROM t2; > +SELECT (SELECT f3a,f3a FROM t3 where f3a > 3) IN (SELECT f1a,f1a FROM t1) FROM t2; > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1) FROM t2; > +SELECT (SELECT f3a, f3b FROM t3 where f3a > 3) IN (SELECT f1a, f1b FROM t1); > + > +# other row operation with NULL single row subquery also should work > +select (null, null) = (null, null); > +SELECT (SELECT f3a, f3a FROM t3 where f3a > 3) = (0, 0); > + > +drop tables t1,t2,t3; > +--echo # return optimizer switch changed in the beginning of this test > set optimizer_switch=@subselect_tmp; > > === modified file 'sql/item.cc' > --- a/sql/item.cc 2011-11-02 08:05:07 +0000 > +++ b/sql/item.cc 2011-11-22 23:46:21 +0000 > @@ -8201,6 +8201,20 @@ void Item_cache::print(String *str, enum > str->append(')'); > } > > +/** > + Assign to this cache NULL value if it is possible > +*/ > + > +void Item_cache::set_null() > +{ > + if (maybe_null) > + { > + null_value= TRUE; > + value_cached= TRUE; > + } > +} > + > + > bool Item_cache_int::cache_value() > { > if (!example) > @@ -8671,6 +8685,20 @@ void Item_cache_row::bring_value() > } > > > +/** > + Assign to this cache NULL value if it is possible > +*/ > + > +void Item_cache_row::set_null() > +{ > + Item_cache::set_null(); > + if (!values) > + return; > + for (uint i= 0; i < item_count; i++) > + values[i]->set_null(); > +}; > + > + > Item_type_holder::Item_type_holder(THD *thd, Item *item) > :Item(thd, item), enum_set_typelib(0), fld_type(get_real_type(item)) > { > > === modified file 'sql/item.h' > --- a/sql/item.h 2011-11-14 17:24:36 +0000 > +++ b/sql/item.h 2011-11-22 23:46:21 +0000 > @@ -3651,6 +3651,7 @@ public: > return false; > return example->is_expensive_processor(arg); > } > + virtual void set_null(); > }; > > > @@ -3825,6 +3826,7 @@ public: > DBUG_VOID_RETURN; > } > bool cache_value(); > + virtual void set_null(); > }; > > > > === modified file 'sql/item_subselect.cc' > --- a/sql/item_subselect.cc 2011-11-21 14:56:32 +0000 > +++ b/sql/item_subselect.cc 2011-11-22 23:46:21 +0000 > @@ -893,7 +893,10 @@ void Item_singlerow_subselect::reset() > { > Item_subselect::reset(); > if (value) > - value->null_value= TRUE; > + { > + for(uint i= 0; i < engine->cols(); i++) > + row[i]->set_null(); > + } > } > > > @@ -1004,6 +1007,11 @@ void Item_singlerow_subselect::fix_lengt > */ > if (engine->no_tables()) > maybe_null= engine->may_be_null(); > + else > + { > + for (uint i= 0; i < max_columns; i++) > + row[i]->maybe_null= TRUE; > + } > } > > > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] WL#244 New (by Knielsen): Reduced table/row locking for SELECT statement with read-only stored function
by worklog-noreply＠askmonty.org 25 Nov '11

25 Nov '11

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Reduced table/row locking for SELECT statement with read-only stored function CREATION DATE..: Thu, 24 Nov 2011, 10:55 SUPERVISOR.....: Monty IMPLEMENTOR....: Monty COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 244 (http://askmonty.org/worklog/?tid=244) VERSION........: Server-5.5 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: When statement (or mixed) binlogging is used, the server currently is overly conservative in its use of table/row locks for SELECT statements that call stored functions. For example: delimiter // CREATE FUNCTION `test_f`(x CHAR(8)) RETURNS INT READS SQL DATA BEGIN DECLARE output INT DEFAULT NULL; SELECT id FROM t1 WHERE a = x INTO output; RETURN output; END // delimiter ; SELECT test_f("foo"); In general, it is possible for a SELECT statement that calls a stored function to modify tables, and thus to need to be binlogged. In statement-based binlogging, this requires taking extra read locks to ensure that replication will apply events from concurrent queries on the master in the correct order, just like normal UPDATE statements. However, most uses of stored functions in SELECT do _not_ modify data, as in the above example. Thus there is no need for extra read locks in statement-based binlog mode, as nothing will be logged to the binlog anyway for the SELECT statement. However, the server currently takes the pessimistic safe route and _always_ takes extra read locks in statement-based binlogging when stored functions are used. There is no analysis to detect that stored functions used are read-only, and hence that no extra locking is needed. This worklog is about improving this so that in common cases where no table modification is done in stored functions, no extra locking is done, while still preserving correct operation for statement-based replication when modification _is_ done. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v4.0.0)

2 1

Re: [Maria-developers] Fwd: [Commits] Rev 3296: Fix bug lp:833777 in file:///home/tsk/mprog/src/5.3-lpb833777/
by Sergei Petrunia 21 Nov '11

21 Nov '11

Hi Timour, On Fri, Nov 18, 2011 at 11:16:27PM +0200, Timour Katchaounov wrote: > Sergey, > > Please review the following patch. > Do you have any idea about the change in > func_compress.result ? I've seen that before, > but don't remember what was the issue. I think in this case it is fairly clear - a warning is produced every time a constant expression is evaluated. Your patch makes one less evaluation, so one fewer warning is emitted. The patch is ok to push after the below notes are addressed. > ------------------------------------------------------------ > revno: 3296 > revision-id: timour(a)askmonty.org-20111118201920-fac3c9870u7eh898 > parent: igor(a)askmonty.org-20111118173551-78xd3uuu5h25756u > fixes bug(s): https://launchpad.net/bugs/833777 > committer: timour(a)askmonty.org > branch nick: 5.3-lpb833777 > timestamp: Fri 2011-11-18 22:19:20 +0200 > message: > Fix bug lp:833777 > > Analysis: > The optimizer distinguishes two kinds of 'constant' conditions: > expensive ones, and non-expensive ones. The non-expensive conditions > are evaluated inside make_join_select(), and if false, already the > optimizer detects empty query results. > > In order to avoid arbitrarily expensive optimization, the evaluation of > expensive constant conditions is delayed until execution. These conditions > are attached to JOIN::exec_const_cond and evaluated in the beginning of > JOIN::exec. The relevant execution logic is: > > JOIN::exec() > { > if (! join->exec_const_cond->val_int()) > { > produce an empty result; > stop execution > } > continue execution > execute the original WHERE clause (that contains exec_const_cond) > ... > } > > As a result, when an expensive constant condition is > TRUE, it is evaluated twice - once through > JOIN::exec_const_cond, and once through JOIN::cond. > When the expensive constant condition is a subquery, > predicate, the subquery is evaluated twice. If we have > many levels of subqueries, this logic results in a chain > of recursive subquery executions that walk a perfect > binary tree. The result is that for subquries with depth N, > JOIN::exec is executed O(2^N) times. > > Solution: > Notice that the second execution of the constant conditions > happens inside do_select(), in the branch: > if (join->table_count == join->const_tables) { ... } > In this case exec_const_cond is equivalent to the whole WHERE > clause, therefore the WHERE clause has already been checked in > the beginnig of JOIN::exec, and has been found to be true. > The bug is addressed by not evaluating the WHERE clause if there > was exec_const_conds, and it was TRUE. > === modified file 'mysql-test/r/func_compress.result' > --- a/mysql-test/r/func_compress.result 2011-02-03 15:00:28 +0000 > +++ b/mysql-test/r/func_compress.result 2011-11-18 20:19:20 +0000 > @@ -103,7 +103,6 @@ foo > Warnings: > Error 1259 ZLIB: Input data corrupted > Error 1259 ZLIB: Input data corrupted > -Error 1259 ZLIB: Input data corrupted > explain select *, uncompress(a) from t1; > id select_type table type possible_keys key key_len ref rows Extra > 1 SIMPLE t1 system NULL NULL NULL NULL 1 > > === modified file 'mysql-test/r/subselect.result' > --- a/mysql-test/r/subselect.result 2011-11-15 21:03:00 +0000 > +++ b/mysql-test/r/subselect.result 2011-11-18 20:19:20 +0000 > @@ -5665,4 +5665,14 @@ ERROR 21000: Subquery returns more than > SET SESSION sql_mode=@old_sql_mode; > DEALLOCATE PREPARE stmt; > DROP TABLE t1; > +# > +# LP BUG#833777 Performance regression with deeply nested subqueries > +# > +create table t1 (a int not null, b char(10) not null); > +insert into t1 values (1, 'a'); > +set @@optimizer_switch='in_to_exists=on,semijoin=off,materialization=off,subquery_cache=off'; > +select benchmark(1, (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1)))))))))))))))))))))))))))))))))); > +benchmark(1, (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a > +0 Why use 'benchmark' function? It seems totally redundant. Please change the test to look like: select (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in ... (select a from t1) ))))))))))))))))))))))))))))))))) AS foo; to make it less cluttered. > +drop table t1; > set optimizer_switch=@subselect_tmp; > > === modified file 'mysql-test/r/subselect_no_mat.result' > --- a/mysql-test/r/subselect_no_mat.result 2011-11-15 21:03:00 +0000 > +++ b/mysql-test/r/subselect_no_mat.result 2011-11-18 20:19:20 +0000 > @@ -5670,6 +5670,16 @@ ERROR 21000: Subquery returns more than > SET SESSION sql_mode=@old_sql_mode; > DEALLOCATE PREPARE stmt; > DROP TABLE t1; > +# > +# LP BUG#833777 Performance regression with deeply nested subqueries > +# > +create table t1 (a int not null, b char(10) not null); > +insert into t1 values (1, 'a'); > +set @@optimizer_switch='in_to_exists=on,semijoin=off,materialization=off,subquery_cache=off'; > +select benchmark(1, (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1)))))))))))))))))))))))))))))))))); > +benchmark(1, (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a > +0 > +drop table t1; > set optimizer_switch=@subselect_tmp; > set optimizer_switch=default; > select @@optimizer_switch like '%materialization=on%'; > > === modified file 'mysql-test/r/subselect_no_opts.result' > --- a/mysql-test/r/subselect_no_opts.result 2011-11-15 21:03:00 +0000 > +++ b/mysql-test/r/subselect_no_opts.result 2011-11-18 20:19:20 +0000 > @@ -5666,5 +5666,15 @@ ERROR 21000: Subquery returns more than > SET SESSION sql_mode=@old_sql_mode; > DEALLOCATE PREPARE stmt; > DROP TABLE t1; > +# > +# LP BUG#833777 Performance regression with deeply nested subqueries > +# > +create table t1 (a int not null, b char(10) not null); > +insert into t1 values (1, 'a'); > +set @@optimizer_switch='in_to_exists=on,semijoin=off,materialization=off,subquery_cache=off'; > +select benchmark(1, (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1)))))))))))))))))))))))))))))))))); > +benchmark(1, (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a > +0 > +drop table t1; > set optimizer_switch=@subselect_tmp; > set @optimizer_switch_for_subselect_test=null; > > === modified file 'mysql-test/r/subselect_no_scache.result' > --- a/mysql-test/r/subselect_no_scache.result 2011-11-15 21:03:00 +0000 > +++ b/mysql-test/r/subselect_no_scache.result 2011-11-18 20:19:20 +0000 > @@ -5669,6 +5669,16 @@ ERROR 21000: Subquery returns more than > SET SESSION sql_mode=@old_sql_mode; > DEALLOCATE PREPARE stmt; > DROP TABLE t1; > +# > +# LP BUG#833777 Performance regression with deeply nested subqueries > +# > +create table t1 (a int not null, b char(10) not null); > +insert into t1 values (1, 'a'); > +set @@optimizer_switch='in_to_exists=on,semijoin=off,materialization=off,subquery_cache=off'; > +select benchmark(1, (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1)))))))))))))))))))))))))))))))))); > +benchmark(1, (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a > +0 > +drop table t1; > set optimizer_switch=@subselect_tmp; > set optimizer_switch=default; > select @@optimizer_switch like '%subquery_cache=on%'; > > === modified file 'mysql-test/r/subselect_no_semijoin.result' > --- a/mysql-test/r/subselect_no_semijoin.result 2011-11-15 21:03:00 +0000 > +++ b/mysql-test/r/subselect_no_semijoin.result 2011-11-18 20:19:20 +0000 > @@ -5666,5 +5666,15 @@ ERROR 21000: Subquery returns more than > SET SESSION sql_mode=@old_sql_mode; > DEALLOCATE PREPARE stmt; > DROP TABLE t1; > +# > +# LP BUG#833777 Performance regression with deeply nested subqueries > +# > +create table t1 (a int not null, b char(10) not null); > +insert into t1 values (1, 'a'); > +set @@optimizer_switch='in_to_exists=on,semijoin=off,materialization=off,subquery_cache=off'; > +select benchmark(1, (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1)))))))))))))))))))))))))))))))))); > +benchmark(1, (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a > +0 > +drop table t1; > set optimizer_switch=@subselect_tmp; > set @optimizer_switch_for_subselect_test=null; > > === modified file 'mysql-test/t/subselect.test' > --- a/mysql-test/t/subselect.test 2011-11-15 21:03:00 +0000 > +++ b/mysql-test/t/subselect.test 2011-11-18 20:19:20 +0000 > @@ -4768,4 +4768,14 @@ SET SESSION sql_mode=@old_sql_mode; > DEALLOCATE PREPARE stmt; > DROP TABLE t1; > > +--echo # > +--echo # LP BUG#833777 Performance regression with deeply nested subqueries > +--echo # > + > +create table t1 (a int not null, b char(10) not null); > +insert into t1 values (1, 'a'); > +set @@optimizer_switch='in_to_exists=on,semijoin=off,materialization=off,subquery_cache=off'; > +select benchmark(1, (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1 where a in (select a from t1)))))))))))))))))))))))))))))))))); > +drop table t1; > + > set optimizer_switch=@subselect_tmp; > > === modified file 'sql/sql_select.cc' > --- a/sql/sql_select.cc 2011-11-15 21:03:00 +0000 > +++ b/sql/sql_select.cc 2011-11-18 20:19:20 +0000 > @@ -14782,9 +14782,15 @@ do_select(JOIN *join,List<Item> *fields, > { > /* > HAVING will be checked after processing aggregate functions, > - But WHERE should checkd here (we alredy have read tables) > + But WHERE should checkd here (we alredy have read tables). > + If there is join->exec_const_cond, and all tables are constant, then it > + is equivalent to join->conds. exec_const_cond is already checked in the > + beginning of JOIN::exec. If it is false, JOIN::exec returns zero > + result already there, therefore execution reaches this point only if > + exec_const_cond is TRUE. Since it is equvalent to join->conds, then > + join->conds is also TRUE. > */ > - if (!join->conds || join->conds->val_int()) > + if (!join->conds || join->exec_const_cond || join->conds->val_int()) > { > error= (*end_select)(join, 0, 0); > if (error == NESTED_LOOP_OK || error == NESTED_LOOP_QUERY_LIMIT) > > _______________________________________________ > commits mailing list > commits(a)mariadb.org > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits -- BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

[Maria-developers] feedback plugin and 5.1 debs
by Kristian Nielsen 16 Nov '11

16 Nov '11

Hi Serg, If you remember, in Athens, we discovered that the feedback plugin is missing from 5.1 .debs. I fixed and pushed this (in the ourdelta scripts). I restarted latest 5.1 buildbot for this, and it seems to work. So just a FYI ... - Kristian.

3 3

Re: [Maria-developers] [Commits] Rev 3276: Fix MySQL BUG#12329653 in file:///home/tsk/mprog/src/5.3/
by Oleksandr Byelkin 12 Nov '11

12 Nov '11

Hi! 12 нояб. 2011, в 10:24, <timour(a)askmonty.org> написал(а): The patch is OK to push after one small change: > At file:///home/tsk/mprog/src/5.3/ > > ------------------------------------------------------------ > revno: 3276 > revision-id: timour(a)askmonty.org-20111112082415-7t6jjzdfg35la7aq > parent: igor(a)askmonty.org-20111111225326-k11e0durn6kvivly > committer: timour(a)askmonty.org > branch nick: 5.3 > timestamp: Sat 2011-11-12 10:24:15 +0200 > message: > Fix MySQL BUG#12329653 > > In MariaDB, when running in ONLY_FULL_GROUP_BY mode, > the server produced in incorrect error message that there > is an aggregate function without GROUP BY, for artificially > created MIN/MAX functions during subquery MIN/MAX optimization. > > The fix introduces a way to distinguish between artifially > created MIN/MAX functions as a result of a rewrite, and normal > ones present in the query. The test for ONLY_FULL_GROUP_BY violation > now tests in addition if a MIN/MAX function was part of a MIN/MAX > subquery rewrite. > > In order to be able to distinguish these MIN/MAX functions, the > patch introduces an additional flag in Item_in_subselect::in_strategy - > SUBS_STRATEGY_CHOSEN. This flag is set when the optimizer makes its > final choice of a subuqery strategy. In order to make the choice > consistent, access to Item_in_subselect::in_strategy is provided > via new class methods. > ****** > Fix MySQL BUG#12329653 > > In MariaDB, when running in ONLY_FULL_GROUP_BY mode, > the server produced in incorrect error message that there > is an aggregate function without GROUP BY, for artificially > created MIN/MAX functions during subquery MIN/MAX optimization. > > The fix introduces a way to distinguish between artifially > created MIN/MAX functions as a result of a rewrite, and normal > ones present in the query. The test for ONLY_FULL_GROUP_BY violation > now tests in addition if a MIN/MAX function was part of a MIN/MAX > subquery rewrite. > > In order to be able to distinguish these MIN/MAX functions, the > patch introduces an additional flag in Item_in_subselect::in_strategy - > SUBS_STRATEGY_CHOSEN. This flag is set when the optimizer makes its > final choice of a subuqery strategy. In order to make the choice > consistent, access to Item_in_subselect::in_strategy is provided > via new class methods. > [skip] > @@ -2606,9 +2606,8 @@ bool > Item_allany_subselect::select_transformer(JOIN *join) > { > DBUG_ENTER("Item_allany_subselect::select_transformer"); > - DBUG_ASSERT((in_strategy & ~(SUBS_MAXMIN_INJECTED | SUBS_MAXMIN_ENGINE | > - SUBS_IN_TO_EXISTS)) == 0); > - in_strategy|= SUBS_IN_TO_EXISTS; > +// DBUG_ASSERT((in_strategy & ~(SUBS_MAXMIN_INJECTED | SUBS_MAXMIN_ENGINE | > +// SUBS_IN_TO_EXISTS)) == 0); > if (upper_item) > upper_item->show= 1; > DBUG_RETURN(select_in_like_transformer(join)); > @@ -2617,7 +2616,7 @@ Item_allany_subselect::select_transforme Remove the comment, but better change it to: DBUG_ASSERT((in_strategy & ~(SUBS_MAXMIN_INJECTED | SUBS_MAXMIN_ENGINE | SUBS_IN_TO_EXISTS | SUBS_STRATEGY_CHOSEN)) == 0); [skip]

1 0

[Maria-developers] Shared plugin library build question
by Steve Ellcey 12 Nov '11

12 Nov '11

I am looking at a difference in the HP-UX vs. Linux mariadb build and was wondering if someone could help me understand why something is set up the way it is. The problem I see is that on HP-UX, when I do a mariadb build I do not get shared plugin libraries like ha_federated. Looking into why this was happening I found that when the plugin libraries like ha_federated are built they have a dependency on libsqlservices and libsqlservices is built only as an archive library. HP-UX doesn't support building shared libaries that depend on archive libraries and so the build of the shared plugin libraries fails when no shared version of libsqlservices is found. So my main question is: why is libsqlservices built only as an archive library? The other question is: if libsqlservices has to be an archive only library, is there is another way to link it in to executables, such as linking it in explicitly when building an executable instead of counting on the shared plugin libraries to bring it in indirectly by making it a dependent library? Steve Ellcey sje(a)cup.hp.com

3 29

Re: [Maria-developers] Shared plugin library build question
by Sergei Golubchik 09 Nov '11

09 Nov '11

Hi, Michael! On Nov 09, Michael Widenius wrote: > > Hi! > > >>>>> "Vladislav" == Vladislav Vaintroub <wlad(a)montyprogram.com> writes: > > <cut> > > > I think the best option for MySQL, and for MariaDB is to rename own base64.h and function prototypes in a way that it does not conflict with system headers on any system it is meant to be run on. Relying on the include order is fragile, I also do not see how it would not solve the problem with same function name different prototypes - how can we always reliably force use our base64_encode rather than system one ? > > Yes, we should rename it to my_base64.h in MySQL 5.5 > > Serg, can you do that for your next push to 5.5? Yes, but this is just one of the problems that was mentioned in this thread. I'll wait till the list is complete and 5.5 compiles. Regards, Sergei

1 0

[Maria-developers] FYI: @@optimizer_switch default changed to have semijoin switched ON.
by Sergei Petrunia 08 Nov '11

08 Nov '11

Hi! If you haven't noticed: last week I've pushed a change that enabled semi-join optimization flags by default, i.e. now the default @@optimizer_swich value has: semijoin=on firstmatch=on loosescan=on Philip: if you have any testing planned for "after semi-join optimizations are enabled by default", now it's time to do them. BR Sergei -- Sergei Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

3 2

[Maria-developers] Bug in ENUM options handling?
by Steve Ellcey 04 Nov '11

04 Nov '11

While working on MariaDB on HP-UX we noticed an inconsistency involving the type of enum arguments. In init_variables and init_one_value (mysys/my_getopt.c), ENUM options are treated as unsigned longs but the enum arguments are actually declared as uint instead of ulong. While this doesn't matter in 32 bit mode when int's and longs's are the same size, it can cause problems in 64 bit mode when long's are 64 bits and int's 32 bits. We found 4 variables that we think should be ulong instead of uint: plugin_maturity plugin_maturity_map thread_handling delay_key_write_options We also changed the value field in the sys_var_enum class (sql/set_var.h) from a uint pointer to a ulong pointer and changed the sys_var_enum and sys_var_enum_const constructors to use a ulong pointer argument for value_arg to make everything consistent. I suppose init_variables and init_one_value could be changed to treat ENUM's as uint instead of ulong but the use of ulong seemed to be intentional so we didn't try making that change as an alternative fix. Is this a change that should be made in the official MariaDB sources? Steve Ellcey sje(a)cup.hp.com

5 8

[Maria-developers] review: [Commits] Rev 3045: Fix gcc 4.6 warning after merge with 5.1 in file:///home/bell/maria/bzr/work-maria-5.2-merge/
by Michael Widenius 04 Nov '11

04 Nov '11

Hi! Review follows: > Fix gcc 4.6 warning after merge with 5.1 > === modified file 'sql/table.cc' > --- a/sql/table.cc 2011-09-01 18:18:29 +0000 > +++ b/sql/table.cc 2011-10-27 16:18:25 +0000 > @@ -5484,7 +5484,7 @@ int update_virtual_fields(THD *thd, TABL > { > DBUG_ENTER("update_virtual_fields"); > Field **vfield_ptr, *vfield; > - int error= 0; > + int error __attribute__ ((unused))= 0; > if (!table || !table->vfield) > DBUG_RETURN(0); The above is wrong. The bug is that we ignore the value of error later down: error= vfield->vcol_info->expr_item->save_in_field(vfield, 0); We should change the above code to be: if (!ignore_errors && error < 0) { my_printf_error(ER_UNKNOWN_ERROR, "Got error when computing value for virtual column %s", vfield->field_name, MYF(0)); DBUG_RETURN(1); } We also need to add 'ignore_errors' as a parameter and add the new parameter to the calls. This can for now be '0' for all cases except for the cases where we check the value for the function, like in fill_record where we already handle the ignore_errors variable. Regards, Monty

1 0

[Maria-developers] Ubuntu 11.10 "Oneiric" packages
by Kristian Nielsen 02 Nov '11

02 Nov '11

Hi Daniel, I believe I've fixed the issues for our .debs on Ubuntu 11.10. I made buildbot create packages for MariaDB 5.2.9. They pass the tests. Buildnumber: 1325 Location in archive: /archive/pack/5.2/build-1325/kvm-deb-oneiric-* Maybe you can make them available alongside the existing 5.2.9 packages? It is the exact code version as the other platforms, but the packaging scripts are modified of course. There is no apparmor profile by default (this kept causing new problems, and future packages on Ubuntu will have it disabled by default). Maybe you can help test them as an extra check on top of the automatic ones in buildbot? I also tried getting 5.3 packages built, but Buildbot kept building the latest revision rather than the 5.3.2 one. I can probably solve that somehow, but unless it is urgent I would like to just wait for the next 5.3 release, which will get Oneiric packages automatically. Note that for 5.1, we currently have a problem, as Oneiric has MySQL 5.1.58 while the MariaDB version is 5.1.55. But for the next 5.1 release we will presumably merge MySQL 5.1.58 anyway, so this will get solved and most people will want to use 5.2 anyway I think. I think that's it, let me know if you have questions or other suggestions. - Kristian.

2 4

Re: [Maria-developers] How you build the MariaDB packages for Ubuntu
by Kristian Nielsen 02 Nov '11

02 Nov '11

Marian Marinov <mm(a)yuhu.biz> writes: > I'm trying to build MariaDB packages for CentOS. > But the problem I have is that the Policy of the distro does not allow > packages that are drop-in replacements of core packages. > Meaning that I can not simply build MariaDB which will provide libmysqlclient > and mysql binaries. > > Can you share with me what is the situation over in the Debian/Ubuntu camp ? In Debian/Ubuntu, there are no "Official" MariaDB packages yet. The ones we provide do replace the standard mysql packages, which is possible since there are no policies enforced for 3rd-party repositories. It will need to be fixed eventually, of course. The normal way I think in Debian is to have MariaDB and MySQL as alternatives, just like Some versions of Ubuntu/Debian have mysql-5.0 and mysql-5.1 as alternatives (can install one or the other, but not both). It is not 100% clear to me what your situation is (I'm not much familiar with CentOS or RPM). Are you saying that the MariaDB packages you create *must* be possible to install alongside the Official CentOS packages? In that case I suppose you have little option but to run mariadb on a different port, different data directory, different socket, different binary names. It seems to be sub-optimal for users, as they would need to reconfigure anything they want to use with MariaDB away from the standard port/names in the MySQL world. Still, this is perfectly possible to do. Configure a non-standard port in your my.cnf, build and install mariadb with a different --prefix, /opt/mariadb or whatever. - Kristian.

3 2

Re: [Maria-developers] [Commits] Rev 3262: Fix bug lp:833702 in file:///home/tsk/mprog/src/5.3/
by Oleksandr Byelkin 01 Nov '11

01 Nov '11

Hi! 01.11.2011 18:19, timour(a)askmonty.org пишет: > At file:///home/tsk/mprog/src/5.3/ > > ------------------------------------------------------------ > revno: 3262 > revision-id: timour(a)askmonty.org-20111101161919-hukh3tipjms3lfzf > parent: sanja(a)askmonty.org-20111101112209-n4ghsy9hpliiassm > fixes bug(s): https://launchpad.net/bugs/833702 > committer: timour(a)askmonty.org > branch nick: 5.3 > timestamp: Tue 2011-11-01 18:19:19 +0200 > message: > Fix bug lp:833702 > > Analysis: > Equality propagation propagated the constant '7' into > args[0] of the Item_in_optimizer that stands for the > "< ANY" predicate. At the same the min/max subquery > rewrite swapped the order of the left and right operands > of the "<" predicate, but used Item_in_subselect::left_expr. > > As a result, when the<ANY predicate is executed early in the > execution phase as a contant condition, instead of a constant > right (swapped) argument of the< predicate, there was a field > (t3.a). This field had no data, since the whole predicate is > considered constant, and it is evaluated before any tables are > read. Having junk in the field row buffer produced wrong result > > Solution: > Fix create_swap to pick the correct Item_in_optimizer left > argument. OK to push.

1 0